-- Logs begin at Fri 2022-08-05 09:46:36 UTC, end at Fri 2022-08-05 10:53:49 UTC. -- Aug 05 10:12:25.739253 dv-cinder-815582-15-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 05 10:12:25.750860 dv-cinder-815582-15-iscsi devstack@keystone.service[80131]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 05 10:12:25.754215 dv-cinder-815582-15-iscsi devstack@keystone.service[80131]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Aug 5 10:12:25 2022] *** Aug 05 10:12:25.754215 dv-cinder-815582-15-iscsi devstack@keystone.service[80131]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 05 10:12:25.754215 dv-cinder-815582-15-iscsi devstack@keystone.service[80131]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 05 10:12:25.754215 dv-cinder-815582-15-iscsi devstack@keystone.service[80131]: nodename: dv-cinder-815582-15-iscsi Aug 05 10:12:25.754215 dv-cinder-815582-15-iscsi devstack@keystone.service[80131]: machine: x86_64 Aug 05 10:12:25.754215 dv-cinder-815582-15-iscsi devstack@keystone.service[80131]: clock source: unix Aug 05 10:12:25.754215 dv-cinder-815582-15-iscsi devstack@keystone.service[80131]: pcre jit disabled Aug 05 10:12:25.754215 dv-cinder-815582-15-iscsi devstack@keystone.service[80131]: detected number of CPU cores: 4 Aug 05 10:12:25.754215 dv-cinder-815582-15-iscsi devstack@keystone.service[80131]: current working directory: / Aug 05 10:12:25.754215 dv-cinder-815582-15-iscsi devstack@keystone.service[80131]: detected binary path: /usr/bin/uwsgi-core Aug 05 10:12:25.754215 dv-cinder-815582-15-iscsi devstack@keystone.service[80131]: your processes number limit is 47680 Aug 05 10:12:25.754215 dv-cinder-815582-15-iscsi devstack@keystone.service[80131]: your memory page size is 4096 bytes Aug 05 10:12:25.754215 dv-cinder-815582-15-iscsi devstack@keystone.service[80131]: detected max file descriptor number: 2048 Aug 05 10:12:25.754215 dv-cinder-815582-15-iscsi devstack@keystone.service[80131]: lock engine: pthread robust mutexes Aug 05 10:12:25.754895 dv-cinder-815582-15-iscsi devstack@keystone.service[80131]: thunder lock: enabled Aug 05 10:12:25.754895 dv-cinder-815582-15-iscsi devstack@keystone.service[80131]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 05 10:12:25.754895 dv-cinder-815582-15-iscsi devstack@keystone.service[80131]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 05 10:12:25.794714 dv-cinder-815582-15-iscsi devstack@keystone.service[80131]: Python main interpreter initialized at 0x555b7f2b79c0 Aug 05 10:12:25.794714 dv-cinder-815582-15-iscsi devstack@keystone.service[80131]: python threads support enabled Aug 05 10:12:25.794714 dv-cinder-815582-15-iscsi devstack@keystone.service[80131]: your server socket listen backlog is limited to 100 connections Aug 05 10:12:25.794714 dv-cinder-815582-15-iscsi devstack@keystone.service[80131]: your mercy for graceful operations on workers is 90 seconds Aug 05 10:12:25.795151 dv-cinder-815582-15-iscsi devstack@keystone.service[80131]: mapped 403077 bytes (393 KB) for 2 cores Aug 05 10:12:25.795151 dv-cinder-815582-15-iscsi devstack@keystone.service[80131]: *** Operational MODE: preforking *** Aug 05 10:12:25.795777 dv-cinder-815582-15-iscsi devstack@keystone.service[80131]: *** uWSGI is running in multiple interpreter mode *** Aug 05 10:12:25.795777 dv-cinder-815582-15-iscsi devstack@keystone.service[80131]: spawned uWSGI master process (pid: 80131) Aug 05 10:12:25.795996 dv-cinder-815582-15-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 05 10:12:25.800431 dv-cinder-815582-15-iscsi devstack@keystone.service[80131]: spawned uWSGI worker 1 (pid: 80141, cores: 1) Aug 05 10:12:25.800431 dv-cinder-815582-15-iscsi devstack@keystone.service[80131]: spawned uWSGI worker 2 (pid: 80142, cores: 1) Aug 05 10:12:25.800431 dv-cinder-815582-15-iscsi devstack@keystone.service[80131]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 05 10:12:31.233249 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 05 10:12:31.233249 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: from cryptography.utils import int_from_bytes Aug 05 10:12:31.233981 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 05 10:12:31.233981 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: from cryptography.utils import int_from_bytes Aug 05 10:12:31.242299 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 05 10:12:31.242299 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: from cryptography.utils import int_from_bytes Aug 05 10:12:31.242816 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 05 10:12:31.242816 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: from cryptography.utils import int_from_bytes Aug 05 10:12:31.283254 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 05 10:12:31.283448 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 05 10:12:31.283575 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] command line args: [] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 05 10:12:31.283880 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 05 10:12:31.284012 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ================================================================================ {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 05 10:12:31.284290 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] admin_token = **** {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.284532 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] config_dir = [] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.284697 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.284903 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] config_source = [] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.285033 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] debug = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.285258 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.285618 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.285764 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.286016 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.286260 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.286485 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] list_limit = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.286652 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] log_config_append = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.286824 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.287028 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] log_dir = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.287197 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] log_file = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.287448 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.287707 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.287946 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.288174 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.288333 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.288507 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.288749 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.288989 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.289218 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.289446 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.289677 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.289903 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.290144 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.290387 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.290633 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 05 10:12:31.290713 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.290824 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 05 10:12:31.291043 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] command line args: [] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 05 10:12:31.291169 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.40/identity {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.291313 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 05 10:12:31.291392 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] publish_errors = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.291490 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ================================================================================ {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 05 10:12:31.291573 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.291742 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] admin_token = **** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.291826 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.291935 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.292016 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] config_dir = [] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.292162 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.292243 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.292344 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] config_source = [] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.292428 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.292514 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] debug = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.292597 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] standard_threads = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.292713 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.292813 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.292947 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.293047 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.293129 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.293206 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.293367 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.293444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] use_journal = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.293520 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] use_json = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.293615 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.293697 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] use_stderr = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.293843 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] list_limit = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.293926 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] use_syslog = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.294020 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] log_config_append = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.294106 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.294197 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.294304 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.294379 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] log_dir = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.294533 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] log_file = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.294613 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.294770 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.294853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.295033 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.295121 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.295616 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.295705 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.295856 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.295933 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.296021 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.296092 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.296192 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.296322 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.296436 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.296565 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.296681 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.296823 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.297083 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.297398 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.297520 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.297666 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.297859 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.297945 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.298101 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.298177 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.298351 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.298428 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.298598 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.298722 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.298834 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.298933 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.299114 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.299368 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.299449 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.40/identity {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.299564 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] publish_errors = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.299646 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.299798 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.299874 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.300029 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.300107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.300206 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.300311 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.300451 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.300560 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.300685 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.300805 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.300919 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] standard_threads = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.301033 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.301149 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.301261 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.301391 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.301474 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.301567 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.301709 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.301785 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] use_journal = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.301903 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] use_json = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.301985 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.302076 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] use_stderr = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.302188 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.302265 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] use_syslog = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.302416 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.302494 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 10:12:31.302646 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.302725 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.302873 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.303015 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.303137 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.303218 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.303370 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.303448 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.303614 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.303691 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.303851 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.303934 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.304085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.304162 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.304316 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.304392 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.304541 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.304630 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.304767 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.304844 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.304992 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.305069 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.305217 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.305293 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.305449 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.305527 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.305692 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.305767 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.305919 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.305994 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.306153 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.306230 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.306378 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.306455 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.306613 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.306701 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.306845 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.306945 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.307114 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.307190 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.307340 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.307428 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.307575 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.307652 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.307808 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.307891 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.308043 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.308193 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.308267 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.308336 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.308498 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.308575 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.308724 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.308813 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.308959 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.309033 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.309189 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.309263 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.309424 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.309508 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.309652 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.309729 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.309883 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.309959 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.310114 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.310223 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.310327 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.310422 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.310552 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.310652 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.310762 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.310864 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.311052 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.311156 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.311290 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.311369 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.311515 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.311610 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.311743 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.311821 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.311974 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.312051 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.312203 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.312292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.312432 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.312508 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.312868 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.313284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.313362 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.313528 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.313618 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.313774 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.313864 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.314017 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.314134 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.314248 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.314337 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.314481 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.314565 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.314713 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.314791 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.314974 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.315067 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.315231 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.315308 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.315465 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.315548 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.315702 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.315786 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.315937 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.316013 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.316174 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.316249 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.316410 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.316496 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.316642 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.316720 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.316870 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.316947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.317108 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.317198 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.317338 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.317416 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.317567 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.317644 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.317796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.317879 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.318021 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.318098 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.318259 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.318336 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.318491 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.318577 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.318726 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.318801 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.318981 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.319063 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.319235 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.319325 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.319482 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.319558 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.319718 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.319793 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.319949 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] auth.external = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.320103 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.320178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.320248 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.320411 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.320487 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.320647 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.320733 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.320883 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] auth.password = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.321019 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.321096 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] auth.token = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.321166 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.321334 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.321411 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.321567 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.321645 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.321800 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.321876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.322025 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.322100 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.322258 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.322335 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.322493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.322577 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.322717 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.322792 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.322971 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] credential.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.323052 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.323217 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.323296 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.323463 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.323540 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.323697 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.323774 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.323935 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.324010 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.324169 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.324246 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.324399 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.324483 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.324631 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.324708 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.324861 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.324937 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.325091 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.325175 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.325321 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.325400 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.325556 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.325640 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.325784 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.325868 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.326023 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.326103 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.326261 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.326342 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.326509 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.326586 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.326745 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.326829 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] federation.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.327016 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.327097 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.327262 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.327339 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.327503 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.327582 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.327726 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.327804 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] auth.external = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.327953 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.328029 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.328256 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.328347 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.328640 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.328878 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.328960 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.329130 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.329218 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] auth.password = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.329366 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.329445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] auth.token = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.329611 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.329689 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.329844 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.329931 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.330089 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.330168 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.330321 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.330397 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.330556 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.330641 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.330789 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.330868 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.331057 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.331136 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.331287 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.331378 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] credential.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.331522 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.331600 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.331768 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.331846 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.332014 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.332098 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.332241 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.332318 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.332466 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.332544 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.332698 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.332783 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.332938 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.333017 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.333181 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.333248 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.333426 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.333529 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.333674 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.333753 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.333921 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.333998 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.334189 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.334269 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.334421 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.334504 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.334658 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.334737 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.334888 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.335028 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.335145 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.335223 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] federation.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.335372 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.335450 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.335607 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.335687 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.335843 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.335920 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.336078 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.336156 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.336322 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.336402 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.336550 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.336627 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.336794 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.336880 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.337028 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.337111 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.337259 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.337336 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.337502 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.337579 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.337744 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.337823 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.337969 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.338047 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.338203 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.338280 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.338429 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.338509 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.338663 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.338742 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.338911 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.339001 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.339154 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.339233 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.339388 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.339471 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.339624 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.339700 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.339864 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.339943 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.340112 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.340191 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.340340 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.340416 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.340565 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.340645 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.340793 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.340869 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.341039 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.341119 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.341269 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.341347 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.341502 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.341580 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.341734 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.341812 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.341963 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.342105 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.342201 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.342279 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.342454 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.342532 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.342682 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.342762 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.342941 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.343023 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.343177 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.343263 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.343407 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.343488 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.343636 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.343713 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.344009 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.344250 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.344492 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.344492 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.344633 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.344703 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.344857 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.344937 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.345102 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.345182 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.345344 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.345422 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.345585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.345670 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.345828 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.345937 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.346079 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.346183 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.346329 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.346417 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.346562 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.346640 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.346802 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.346879 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.347052 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.347145 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.347287 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.347370 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.347527 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.347638 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.347769 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.347855 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.348007 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.348085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.348238 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.348314 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.348485 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.348569 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.348708 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.348784 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.348950 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.349033 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.349177 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.349261 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.349408 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.349484 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.349640 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.349717 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.349881 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.349967 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.350116 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.350191 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.350340 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.350416 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.350570 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.350653 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.350806 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] resource.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.350882 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.351088 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.351168 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.351310 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.351399 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.351554 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.351631 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.351802 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.351879 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.352047 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.352131 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.352282 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.352357 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.352542 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.352666 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.352742 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.352821 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.352962 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.353041 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.353194 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] role.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.353270 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.353421 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.353505 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] role.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.353654 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.353730 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.353898 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.353981 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.354137 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.354222 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.354380 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.354456 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.354620 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.354696 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.354846 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.354956 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.355119 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.355197 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.355356 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.355433 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.355593 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.355679 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.355820 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.355897 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.356053 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.356129 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.356292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.356376 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.356534 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.356612 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.356766 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.356842 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.357020 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.357174 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.357249 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.357382 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.357480 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.357627 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.357713 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.357872 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.357949 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.358115 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.358193 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.358360 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.358497 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.358577 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.358744 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.358822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.359003 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.359132 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.359268 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.359423 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.359563 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] resource.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.359683 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.359806 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.360017 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.360260 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.360346 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.360460 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.360540 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.360706 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.360785 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.360939 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.361025 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.361171 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.361250 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.361399 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.361478 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.361634 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] token.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.361714 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.361867 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.361952 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.362109 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.362188 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.362342 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.362422 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] role.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.362592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] role.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.362669 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.362825 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.362950 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.363078 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.363159 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.363318 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.363397 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.363558 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.363637 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.363805 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.363885 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.364042 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.364229 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.364349 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.364425 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.364505 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.364578 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.364723 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.364802 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.364966 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.365045 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.365193 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.365273 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.365433 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.365511 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.365677 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.365753 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.365928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.366007 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.connection = **** {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.366196 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.366330 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.366439 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.366572 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.366687 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.366824 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.366951 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.367101 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.367179 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.367350 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.367428 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.367580 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.367658 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.367814 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.367892 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.368057 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.368134 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.368291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.368370 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.368529 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.368604 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.368759 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.368836 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.368991 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.369069 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.369218 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.369295 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.369460 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.369538 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.369687 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.369768 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.369925 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.370003 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.370159 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.370236 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.370315 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 05 10:12:31.370433 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.370605 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.370838 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] token.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.371103 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.371353 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.371588 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.371841 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.372088 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.372325 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.372567 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.372811 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.373057 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.373292 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.373530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.373763 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.374007 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.374256 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.374485 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.374745 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.375106 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.375839 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.connection = **** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.376103 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.376361 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.376608 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.376859 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.377102 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.377343 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.377582 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.377821 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.378069 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.378309 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.378554 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.378793 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.379075 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.379313 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.379562 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.379792 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.380034 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.380269 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 10:12:31.380438 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 05 10:12:31.399563 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.399788 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.400008 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.405460 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.405703 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.405917 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.408532 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.414872 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.438759 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.438992 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.443653 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.443895 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.491124 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.492542 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.541095 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.541293 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.542757 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.543018 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.605755 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.605925 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.606142 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.606225 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.606308 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.606373 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.606530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.606656 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.606725 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.606786 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.606971 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.607192 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.630083 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.630299 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.630457 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.630565 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.630644 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.630706 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.630932 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.631052 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.631140 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.631216 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.631319 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.631407 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.631545 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.631764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.642544 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.643099 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.655201 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.655412 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.655626 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.655774 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.655774 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.655908 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.655980 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.656041 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.656184 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.656313 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.656453 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.656570 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.657207 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.657433 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 10:12:31.673652 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.673842 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 05 10:12:31.674125 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.674295 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 05 10:12:31.674390 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.674583 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.674710 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 05 10:12:31.674812 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.674925 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.675106 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.675247 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 05 10:12:31.675330 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.675394 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.675660 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.675922 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.676174 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.684960 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.685234 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.685499 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.685778 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.685901 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.686000 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.686301 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.686584 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.692792 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 10:12:31.694084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 10:12:31.697481 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 10:12:31.697758 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.698033 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.698307 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.698581 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.698812 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 10:12:31.698995 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.699070 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.699132 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.699367 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.699495 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.699577 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.699656 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.699865 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.699991 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.700134 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.700258 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.700397 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.700665 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.700936 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.701205 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.701467 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.723568 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.725043 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 10:12:31.725320 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.725442 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.726920 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 10:12:31.727203 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.729921 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 10:12:31.730195 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.730475 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.731844 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 10:12:31.732119 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.732401 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.737348 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 10:12:31.737625 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.739533 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 10:12:31.739822 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.741720 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 10:12:31.742001 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.742270 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.742558 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.742857 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.743158 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.743422 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.743691 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.743966 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 10:12:31.744252 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.744515 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.744797 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.745078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.745357 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.745608 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.745871 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.759654 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.759941 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.762493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.762777 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.763875 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 10:12:31.767169 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.767374 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 10:12:31.767474 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.770474 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.770769 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.771849 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 10:12:31.775459 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 10:12:31.775636 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 10:12:31.778355 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 10:12:31.778483 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.778652 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.778917 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.779239 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.779509 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.779781 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.781796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.782855 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.783796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.784086 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.784367 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.784647 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.796250 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 10:12:31.796532 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.796793 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.797204 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.801718 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 10:12:31.802004 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.802273 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.802528 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.804224 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.805763 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.806030 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.808863 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 10:12:31.809092 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.809398 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.809525 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.809672 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.809998 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.810949 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.811263 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.814764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 10:12:31.815075 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.815373 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.815663 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.815937 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.822033 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 10:12:31.822310 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.822587 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.822862 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.823164 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.823438 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.823716 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.827395 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 10:12:31.827674 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.827949 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.828231 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.828996 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.829316 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.829605 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.839333 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 10:12:31.842286 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 10:12:31.845414 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.845800 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 10:12:31.847948 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.848801 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 10:12:31.849421 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 10:12:31.849703 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.849984 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.851776 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.854291 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.855918 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 10:12:31.856234 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.856532 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.856675 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.858128 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 10:12:31.861466 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.861753 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.862025 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.862293 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.863274 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.864896 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 10:12:31.867859 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.868145 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.868429 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.868698 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.870313 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 10:12:31.870606 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.870876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.877416 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 10:12:31.877703 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.877826 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 10:12:31.877964 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.878083 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.878381 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.878642 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.878917 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.879200 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.879468 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.879745 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.880124 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.880392 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.880656 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.880907 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.881167 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.881416 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.884874 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 10:12:31.885167 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.885448 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.885711 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.885979 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.886246 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.886518 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.886800 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.887195 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.887475 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.887747 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.888008 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.888268 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.888534 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 10:12:31.914822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x555b7f2b79c0 pid: 80141 (default app) Aug 05 10:12:31.918150 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accebe7f-bb78-4730-9288-a12330c9df07 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:31.918435 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accebe7f-bb78-4730-9288-a12330c9df07 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:31.918704 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accebe7f-bb78-4730-9288-a12330c9df07 None None] PATH_INFO: `/v3` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:31.923530 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1/1] 10.222.0.40 () {50 vars in 791 bytes} [Fri Aug 5 10:12:31 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:12:31.924229 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x555b7f2b79c0 pid: 80142 (default app) Aug 05 10:12:39.588799 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e10cc-4fc2-4332-b718-c83801a41d9b None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:39.589730 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e10cc-4fc2-4332-b718-c83801a41d9b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:39.589730 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e10cc-4fc2-4332-b718-c83801a41d9b None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:39.591933 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 2/2] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:12:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:12:39.601785 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d57797-8b0a-4d2a-8b7d-2c39532aca25 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:39.602232 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d57797-8b0a-4d2a-8b7d-2c39532aca25 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:39.602373 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d57797-8b0a-4d2a-8b7d-2c39532aca25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:39.682726 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: INFO dbcounter [None req-48d57797-8b0a-4d2a-8b7d-2c39532aca25 None None] Registered counter for database keystone Aug 05 10:12:39.704530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG oslo_db.sqlalchemy.engines [None req-48d57797-8b0a-4d2a-8b7d-2c39532aca25 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80142) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 05 10:12:39.713040 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writer thread running {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 05 10:12:39.903785 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG passlib.handlers.bcrypt [None req-48d57797-8b0a-4d2a-8b7d-2c39532aca25 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80142) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 05 10:12:39.904129 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG passlib.handlers.bcrypt [None req-48d57797-8b0a-4d2a-8b7d-2c39532aca25 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80142) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 05 10:12:39.966002 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-48d57797-8b0a-4d2a-8b7d-2c39532aca25 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:12:39.967623 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-48d57797-8b0a-4d2a-8b7d-2c39532aca25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:12:40.117394 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1/3] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:12:39 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 521 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 05 10:12:40.264876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e10cc-4fc2-4332-b718-c83801a41d9b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:12:40.342054 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: INFO dbcounter [None req-214e10cc-4fc2-4332-b718-c83801a41d9b None None] Registered counter for database keystone Aug 05 10:12:40.365569 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG oslo_db.sqlalchemy.engines [None req-214e10cc-4fc2-4332-b718-c83801a41d9b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80141) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 05 10:12:40.375624 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writer thread running {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 05 10:12:40.500549 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e10cc-4fc2-4332-b718-c83801a41d9b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:12:40.506005 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e0301-4ad6-4363-845b-821403aa4041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:12:40.506640 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e0301-4ad6-4363-845b-821403aa4041 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:40.506941 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e0301-4ad6-4363-845b-821403aa4041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:40.507215 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e0301-4ad6-4363-845b-821403aa4041 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:40.516311 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240e0301-4ad6-4363-845b-821403aa4041 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:12:40.568395 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: WARNING keystone.common.rbac_enforcer.enforcer [None req-240e0301-4ad6-4363-845b-821403aa4041 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 05 10:12:40.568709 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240e0301-4ad6-4363-845b-821403aa4041 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:12:40.579213 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 3/4] 10.222.0.40 () {58 vars in 1324 bytes} [Fri Aug 5 10:12:40 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 10:12:40.585730 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d57797-8b0a-4d2a-8b7d-2c39532aca25 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:12:40.600878 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d57797-8b0a-4d2a-8b7d-2c39532aca25 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:12:40.606394 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2071c8-2b08-4320-b808-d927236f4875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:12:40.607562 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2071c8-2b08-4320-b808-d927236f4875 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:40.607838 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2071c8-2b08-4320-b808-d927236f4875 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:40.608107 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2071c8-2b08-4320-b808-d927236f4875 admin admin] PATH_INFO: `/v3/projects/be89c43472bc49a2bd0e46cddf175958` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:40.610181 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2071c8-2b08-4320-b808-d927236f4875 admin admin] RBAC: Authorizing `identity:get_project(project_id=be89c43472bc49a2bd0e46cddf175958)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:12:40.660360 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: WARNING keystone.common.rbac_enforcer.enforcer [None req-8c2071c8-2b08-4320-b808-d927236f4875 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 05 10:12:40.660669 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2071c8-2b08-4320-b808-d927236f4875 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:12:40.665312 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 2/5] 10.222.0.40 () {56 vars in 1207 bytes} [Fri Aug 5 10:12:40 2022] GET /identity/v3/projects/be89c43472bc49a2bd0e46cddf175958 => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:12:42.640215 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4588c3d-69ef-446e-ba3c-9f438685aa45 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:42.641197 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4588c3d-69ef-446e-ba3c-9f438685aa45 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:42.641197 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4588c3d-69ef-446e-ba3c-9f438685aa45 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:42.648686 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 4/6] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:12:42 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:12:42.655413 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7490d297-b57c-49a7-8b89-1ef929930620 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:42.655881 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7490d297-b57c-49a7-8b89-1ef929930620 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:42.655974 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7490d297-b57c-49a7-8b89-1ef929930620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:42.680736 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-7490d297-b57c-49a7-8b89-1ef929930620 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:12:42.683166 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7490d297-b57c-49a7-8b89-1ef929930620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:12:42.697924 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 3/7] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:12:42 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 05 10:12:42.722504 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4588c3d-69ef-446e-ba3c-9f438685aa45 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:12:42.741447 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4588c3d-69ef-446e-ba3c-9f438685aa45 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:12:42.747933 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c5b10f-ed71-4504-88f8-e874b553a512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:12:42.748544 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5b10f-ed71-4504-88f8-e874b553a512 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:42.748822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5b10f-ed71-4504-88f8-e874b553a512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:42.749095 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5b10f-ed71-4504-88f8-e874b553a512 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:42.757329 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c5b10f-ed71-4504-88f8-e874b553a512 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:12:42.759135 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c5b10f-ed71-4504-88f8-e874b553a512 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:12:42.767858 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 5/8] 10.222.0.40 () {58 vars in 1324 bytes} [Fri Aug 5 10:12:42 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 10:12:42.773472 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7490d297-b57c-49a7-8b89-1ef929930620 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:12:42.782544 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7490d297-b57c-49a7-8b89-1ef929930620 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:12:42.788283 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61845532-fb0f-4348-b04b-b3a8950108d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:12:42.789245 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61845532-fb0f-4348-b04b-b3a8950108d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:42.789603 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61845532-fb0f-4348-b04b-b3a8950108d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:42.789953 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61845532-fb0f-4348-b04b-b3a8950108d9 admin admin] PATH_INFO: `/v3/users/898d0645ae1e477983200e39184fa9ed` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:42.792310 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61845532-fb0f-4348-b04b-b3a8950108d9 admin admin] RBAC: Authorizing `identity:get_user(user_id=898d0645ae1e477983200e39184fa9ed)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:12:42.795235 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61845532-fb0f-4348-b04b-b3a8950108d9 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:12:42.797888 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 4/9] 10.222.0.40 () {56 vars in 1198 bytes} [Fri Aug 5 10:12:42 2022] GET /identity/v3/users/898d0645ae1e477983200e39184fa9ed => generated 247 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:12:44.856797 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda625fc-6a67-42a8-b55c-65bf5d3892b0 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:44.857363 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda625fc-6a67-42a8-b55c-65bf5d3892b0 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:44.857363 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda625fc-6a67-42a8-b55c-65bf5d3892b0 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:44.858656 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 6/10] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:12:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:12:44.864228 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02758297-dad0-447f-9f7e-1833164e3eae None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:44.864671 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02758297-dad0-447f-9f7e-1833164e3eae None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:44.864851 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02758297-dad0-447f-9f7e-1833164e3eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:44.883305 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1168ec1-55fc-41e5-8866-51352a3a3049 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:44.883587 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1168ec1-55fc-41e5-8866-51352a3a3049 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:44.883857 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1168ec1-55fc-41e5-8866-51352a3a3049 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:44.884771 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-02758297-dad0-447f-9f7e-1833164e3eae None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:12:44.885182 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 7/11] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:12:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:12:44.886047 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-02758297-dad0-447f-9f7e-1833164e3eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:12:44.890010 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065a6c5-e7e1-4b11-abdf-49aac24c5b8f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:44.890291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065a6c5-e7e1-4b11-abdf-49aac24c5b8f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:44.890615 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065a6c5-e7e1-4b11-abdf-49aac24c5b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:44.933166 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG passlib.handlers.bcrypt [None req-c065a6c5-e7e1-4b11-abdf-49aac24c5b8f None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80141) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 05 10:12:44.933626 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG passlib.handlers.bcrypt [None req-c065a6c5-e7e1-4b11-abdf-49aac24c5b8f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80141) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 05 10:12:44.968628 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 5/12] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:12:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:12:44.978728 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c065a6c5-e7e1-4b11-abdf-49aac24c5b8f None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:12:44.980348 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c065a6c5-e7e1-4b11-abdf-49aac24c5b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:12:44.990313 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02758297-dad0-447f-9f7e-1833164e3eae None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:12:45.014050 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:12:45.083165 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 8/13] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:12:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:12:45.097032 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02758297-dad0-447f-9f7e-1833164e3eae None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:12:45.108694 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c065a6c5-e7e1-4b11-abdf-49aac24c5b8f None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:12:45.133615 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fa64a9-6560-4844-bc4b-6e588c1c0edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:12:45.134254 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa64a9-6560-4844-bc4b-6e588c1c0edd None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:45.134535 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa64a9-6560-4844-bc4b-6e588c1c0edd None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:45.134800 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa64a9-6560-4844-bc4b-6e588c1c0edd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:45.206044 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fa64a9-6560-4844-bc4b-6e588c1c0edd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:12:45.207583 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fa64a9-6560-4844-bc4b-6e588c1c0edd None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:12:45.215734 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c065a6c5-e7e1-4b11-abdf-49aac24c5b8f None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:12:45.217993 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:12:45.231508 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:12:45.257730 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7dfa1b-001f-49d8-8c25-fd7cf61b52f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:12:45.258873 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7dfa1b-001f-49d8-8c25-fd7cf61b52f1 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:45.259191 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7dfa1b-001f-49d8-8c25-fd7cf61b52f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:45.259466 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7dfa1b-001f-49d8-8c25-fd7cf61b52f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:45.271703 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7dfa1b-001f-49d8-8c25-fd7cf61b52f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:12:45.280346 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-ca7dfa1b-001f-49d8-8c25-fd7cf61b52f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:12:45.280346 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:12:45.280346 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:12:45.280346 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:12:45.280346 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:12:45.280346 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:12:45.280346 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:12:45.280346 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:12:45.280346 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:12:45.280346 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:12:45.280346 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:12:45.280346 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:12:45.280346 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:12:45.280346 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:12:45.280346 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:12:45.280346 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:12:45.281374 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:12:45.281374 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:12:45.281374 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:12:45.281374 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:12:45.281374 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:12:45.281374 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:12:45.281374 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:12:45.281374 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:12:45.281374 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:12:45.281374 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:12:45.281374 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:12:45.281374 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:12:45.281374 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:12:45.281374 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:12:45.281374 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:12:45.281374 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:12:45.282284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:12:45.282284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:12:45.282284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:12:45.282284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:12:45.282284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:12:45.282284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:12:45.282284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:12:45.282284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:12:45.282284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:12:45.282284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:12:45.282284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:12:45.282284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:12:45.282284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 9/14] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 10:12:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:12:45.287674 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7dfa1b-001f-49d8-8c25-fd7cf61b52f1 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:12:45.318860 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 6/15] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:12:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 10:12:45.327233 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fa64a9-6560-4844-bc4b-6e588c1c0edd None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:12:45.400767 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7dfa1b-001f-49d8-8c25-fd7cf61b52f1 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:12:45.429392 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fa64a9-6560-4844-bc4b-6e588c1c0edd None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:12:45.432491 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b699f0c-1ea9-4c38-9150-3d061a98cccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:12:45.433109 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b699f0c-1ea9-4c38-9150-3d061a98cccb None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:45.433388 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b699f0c-1ea9-4c38-9150-3d061a98cccb None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:45.433657 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b699f0c-1ea9-4c38-9150-3d061a98cccb None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:45.434373 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b699f0c-1ea9-4c38-9150-3d061a98cccb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:12:45.434670 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b699f0c-1ea9-4c38-9150-3d061a98cccb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:12:45.436150 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b699f0c-1ea9-4c38-9150-3d061a98cccb None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:12:45.447914 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 10/16] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:12:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:12:45.451372 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:12:45.454291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b699f0c-1ea9-4c38-9150-3d061a98cccb None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:12:45.465113 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d6789-b050-46e8-b220-e7834ae68232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:12:45.465818 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d6789-b050-46e8-b220-e7834ae68232 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:45.466080 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d6789-b050-46e8-b220-e7834ae68232 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:45.466355 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d6789-b050-46e8-b220-e7834ae68232 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:45.477197 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d6789-b050-46e8-b220-e7834ae68232 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:12:45.478688 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d6789-b050-46e8-b220-e7834ae68232 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:12:45.481047 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:12:45.497450 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-ca5d6789-b050-46e8-b220-e7834ae68232 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:12:45.497450 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:12:45.497450 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:12:45.497450 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:12:45.497450 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:12:45.497450 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:12:45.497450 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:12:45.497450 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:12:45.497450 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:12:45.497450 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:12:45.497450 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:12:45.497450 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:12:45.497450 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:12:45.497450 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:12:45.497450 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:12:45.498418 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:12:45.498418 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:12:45.498418 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:12:45.498418 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:12:45.498418 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:12:45.498418 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:12:45.498418 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:12:45.498418 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:12:45.498418 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:12:45.498418 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:12:45.498418 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:12:45.498418 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:12:45.498418 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:12:45.498418 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:12:45.498418 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:12:45.498418 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:12:45.498418 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:12:45.499384 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:12:45.499384 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:12:45.499384 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:12:45.499384 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:12:45.499384 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 7/17] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:12:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:12:45.504635 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d6789-b050-46e8-b220-e7834ae68232 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:12:45.556984 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b699f0c-1ea9-4c38-9150-3d061a98cccb None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:12:45.591706 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd8d3f-5d16-49c6-8628-e404a285ac04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:12:45.592388 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd8d3f-5d16-49c6-8628-e404a285ac04 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:45.592669 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd8d3f-5d16-49c6-8628-e404a285ac04 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:45.592948 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd8d3f-5d16-49c6-8628-e404a285ac04 None admin] PATH_INFO: `/v3/users/898d0645ae1e477983200e39184fa9ed` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:45.594895 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dd8d3f-5d16-49c6-8628-e404a285ac04 None admin] RBAC: Authorizing `identity:get_user(user_id=898d0645ae1e477983200e39184fa9ed)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:12:45.596441 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dd8d3f-5d16-49c6-8628-e404a285ac04 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:12:45.598165 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 11/18] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:12:45 2022] GET /identity/v3/users/898d0645ae1e477983200e39184fa9ed => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:12:45.604217 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd8d3f-5d16-49c6-8628-e404a285ac04 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:12:45.711067 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:12:45.717068 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d6789-b050-46e8-b220-e7834ae68232 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:12:45.718874 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd8d3f-5d16-49c6-8628-e404a285ac04 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:12:45.751010 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa47ec2-3b99-4e2a-af56-0d047140ac42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:12:45.751629 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa47ec2-3b99-4e2a-af56-0d047140ac42 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:45.751910 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa47ec2-3b99-4e2a-af56-0d047140ac42 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:45.752179 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa47ec2-3b99-4e2a-af56-0d047140ac42 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:45.752870 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa47ec2-3b99-4e2a-af56-0d047140ac42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:12:45.753169 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa47ec2-3b99-4e2a-af56-0d047140ac42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:12:45.753942 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6eb406-d1ac-4661-984d-91fee10b27aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:12:45.754553 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa47ec2-3b99-4e2a-af56-0d047140ac42 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:12:45.754652 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6eb406-d1ac-4661-984d-91fee10b27aa None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:45.754864 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6eb406-d1ac-4661-984d-91fee10b27aa None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:45.755171 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6eb406-d1ac-4661-984d-91fee10b27aa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:45.756380 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6eb406-d1ac-4661-984d-91fee10b27aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:12:45.757851 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6eb406-d1ac-4661-984d-91fee10b27aa None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:12:45.759516 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 12/19] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:12:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:12:45.765633 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6eb406-d1ac-4661-984d-91fee10b27aa None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:12:45.771752 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 8/20] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:12:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:12:45.872155 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6eb406-d1ac-4661-984d-91fee10b27aa None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:12:45.910329 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b436a13-717d-47d6-846e-6f2b2f51157d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:12:45.910983 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b436a13-717d-47d6-846e-6f2b2f51157d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:45.911270 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b436a13-717d-47d6-846e-6f2b2f51157d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:45.911543 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b436a13-717d-47d6-846e-6f2b2f51157d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:45.912349 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b436a13-717d-47d6-846e-6f2b2f51157d None admin] RBAC: Adding query filter params (role.id=daef89adcaeb48a382a0bcb03340ccbb, scope.domain.id=default, user.id=898d0645ae1e477983200e39184fa9ed) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:12:45.912655 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b436a13-717d-47d6-846e-6f2b2f51157d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:12:45.914144 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b436a13-717d-47d6-846e-6f2b2f51157d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:12:45.923553 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 13/21] 10.222.0.40 () {56 vars in 1322 bytes} [Fri Aug 5 10:12:45 2022] GET /identity/v3/role_assignments?user.id=898d0645ae1e477983200e39184fa9ed&scope.domain.id=default&role.id=daef89adcaeb48a382a0bcb03340ccbb => generated 233 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:12:48.280261 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812ba20-5be2-48e1-abf6-e73fa00ebd85 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:48.280796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812ba20-5be2-48e1-abf6-e73fa00ebd85 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:48.280796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812ba20-5be2-48e1-abf6-e73fa00ebd85 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:48.282338 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 9/22] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:12:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:12:48.301162 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f69973-3f0a-449d-842e-e4566e4345d2 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:48.301680 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f69973-3f0a-449d-842e-e4566e4345d2 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:48.301680 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f69973-3f0a-449d-842e-e4566e4345d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:48.331939 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-05f69973-3f0a-449d-842e-e4566e4345d2 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:12:48.333151 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-05f69973-3f0a-449d-842e-e4566e4345d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:12:48.349004 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76359f-3ab8-4e1c-98a9-d2a02653d9a4 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:48.349286 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76359f-3ab8-4e1c-98a9-d2a02653d9a4 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:48.349552 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76359f-3ab8-4e1c-98a9-d2a02653d9a4 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:48.351216 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 10/23] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:12:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:12:48.356106 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a6faf1-5ed4-4065-8383-51ba52572c09 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:48.356443 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a6faf1-5ed4-4065-8383-51ba52572c09 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:48.356716 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a6faf1-5ed4-4065-8383-51ba52572c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:48.374012 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:12:48.389522 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-10a6faf1-5ed4-4065-8383-51ba52572c09 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:12:48.390773 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-10a6faf1-5ed4-4065-8383-51ba52572c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:12:48.459306 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:12:48.486498 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 14/24] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:12:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:12:48.520795 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 11/25] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:12:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:12:48.534998 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f69973-3f0a-449d-842e-e4566e4345d2 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:12:48.549303 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc68091-2ce8-4870-9c99-27742cc64bf9 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:48.549595 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc68091-2ce8-4870-9c99-27742cc64bf9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:48.549852 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc68091-2ce8-4870-9c99-27742cc64bf9 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:48.552201 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 12/26] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:12:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:12:48.569507 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933e1a8-53a5-45bd-b2e0-405e954a226d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:48.569811 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933e1a8-53a5-45bd-b2e0-405e954a226d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:48.570053 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933e1a8-53a5-45bd-b2e0-405e954a226d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:48.593996 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6933e1a8-53a5-45bd-b2e0-405e954a226d None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:12:48.595236 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6933e1a8-53a5-45bd-b2e0-405e954a226d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:12:48.661941 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f69973-3f0a-449d-842e-e4566e4345d2 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:12:48.679442 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 13/27] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:12:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:12:48.680040 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6933e1a8-53a5-45bd-b2e0-405e954a226d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:12:48.721531 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b2dbd-a67e-418d-8d16-8c6191bff0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:12:48.722108 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b2dbd-a67e-418d-8d16-8c6191bff0ea None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:48.722390 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b2dbd-a67e-418d-8d16-8c6191bff0ea None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:48.722757 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b2dbd-a67e-418d-8d16-8c6191bff0ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:48.735242 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502b2dbd-a67e-418d-8d16-8c6191bff0ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:12:48.740444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-502b2dbd-a67e-418d-8d16-8c6191bff0ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:12:48.740444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:12:48.740444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:12:48.740444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:12:48.740444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:12:48.740444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:12:48.740444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:12:48.740444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:12:48.740444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:12:48.740444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:12:48.740444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:12:48.740444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:12:48.740444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:12:48.740444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:12:48.740444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:12:48.740444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:12:48.741241 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:12:48.741241 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:12:48.741241 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:12:48.741241 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:12:48.741241 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:12:48.741241 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:12:48.741241 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:12:48.741241 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:12:48.741241 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:12:48.741241 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:12:48.741241 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:12:48.741241 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:12:48.741241 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:12:48.741241 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:12:48.741241 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:12:48.741241 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:12:48.742002 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:12:48.742002 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:12:48.742002 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:12:48.742002 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:12:48.742002 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:12:48.742002 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:12:48.742002 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:12:48.742002 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:12:48.742002 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:12:48.742002 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:12:48.742002 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:12:48.742002 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:12:48.742567 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 15/28] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 10:12:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:12:48.746920 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f6985-8179-4813-bcc6-656acc61c537 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:48.746920 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f6985-8179-4813-bcc6-656acc61c537 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:48.746920 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f6985-8179-4813-bcc6-656acc61c537 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:48.749454 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 16/29] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:12:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:12:48.750888 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3f6985-8179-4813-bcc6-656acc61c537 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:12:48.771321 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:12:48.791274 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:12:48.805692 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6933e1a8-53a5-45bd-b2e0-405e954a226d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:12:48.850915 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245fc8bf-cca0-45bd-99cd-faf3002d7f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:12:48.851796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245fc8bf-cca0-45bd-99cd-faf3002d7f57 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:48.852440 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245fc8bf-cca0-45bd-99cd-faf3002d7f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:48.852804 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245fc8bf-cca0-45bd-99cd-faf3002d7f57 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:48.853523 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245fc8bf-cca0-45bd-99cd-faf3002d7f57 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:12:48.855434 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245fc8bf-cca0-45bd-99cd-faf3002d7f57 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:12:48.869465 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3f6985-8179-4813-bcc6-656acc61c537 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:12:48.908191 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a883b4b9-90d5-4f3c-a09b-dd8e3fe38714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:12:48.908883 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a883b4b9-90d5-4f3c-a09b-dd8e3fe38714 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:48.909171 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a883b4b9-90d5-4f3c-a09b-dd8e3fe38714 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:48.909444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a883b4b9-90d5-4f3c-a09b-dd8e3fe38714 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:48.910215 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a883b4b9-90d5-4f3c-a09b-dd8e3fe38714 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:12:48.911767 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a883b4b9-90d5-4f3c-a09b-dd8e3fe38714 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:12:48.995110 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 14/30] 10.222.0.40 () {60 vars in 1131 bytes} [Fri Aug 5 10:12:48 2022] POST /identity/v3/roles => generated 219 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:12:48.996425 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245fc8bf-cca0-45bd-99cd-faf3002d7f57 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:12:49.118408 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245fc8bf-cca0-45bd-99cd-faf3002d7f57 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:12:49.178473 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84ce59a-fd8b-4ff8-9591-03833c491c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:12:49.179232 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ce59a-fd8b-4ff8-9591-03833c491c7c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:49.179521 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ce59a-fd8b-4ff8-9591-03833c491c7c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:49.179796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ce59a-fd8b-4ff8-9591-03833c491c7c None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:49.180549 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84ce59a-fd8b-4ff8-9591-03833c491c7c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:12:49.180858 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84ce59a-fd8b-4ff8-9591-03833c491c7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:12:49.182516 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84ce59a-fd8b-4ff8-9591-03833c491c7c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:12:49.206937 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 15/31] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:12:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:12:49.212591 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f11203-f412-4385-8dde-ce58fa40c369 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:49.212978 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f11203-f412-4385-8dde-ce58fa40c369 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:49.213256 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f11203-f412-4385-8dde-ce58fa40c369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:49.236707 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 17/32] 10.222.0.40 () {60 vars in 1131 bytes} [Fri Aug 5 10:12:48 2022] POST /identity/v3/roles => generated 225 bytes in 487 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:12:49.238145 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a883b4b9-90d5-4f3c-a09b-dd8e3fe38714 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:12:49.248118 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c3f11203-f412-4385-8dde-ce58fa40c369 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:12:49.250252 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c3f11203-f412-4385-8dde-ce58fa40c369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:12:49.281164 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:12:49.412535 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:12:49.424281 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 16/33] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:12:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:12:49.445638 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a883b4b9-90d5-4f3c-a09b-dd8e3fe38714 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:12:49.458174 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f11203-f412-4385-8dde-ce58fa40c369 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:12:49.505491 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdf51eb-f356-4402-bbe4-5a8b1ca01edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:12:49.506084 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf51eb-f356-4402-bbe4-5a8b1ca01edd None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:49.506367 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf51eb-f356-4402-bbe4-5a8b1ca01edd None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:49.506645 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf51eb-f356-4402-bbe4-5a8b1ca01edd None admin] PATH_INFO: `/v3/users/898d0645ae1e477983200e39184fa9ed` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:49.507973 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdf51eb-f356-4402-bbe4-5a8b1ca01edd None admin] RBAC: Authorizing `identity:get_user(user_id=898d0645ae1e477983200e39184fa9ed)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:12:49.509559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdf51eb-f356-4402-bbe4-5a8b1ca01edd None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:12:49.512527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 18/34] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:12:49 2022] GET /identity/v3/users/898d0645ae1e477983200e39184fa9ed => generated 247 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:12:49.513797 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6e410-9c30-42d5-8f28-d583e8ab9a38 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:49.514264 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6e410-9c30-42d5-8f28-d583e8ab9a38 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:49.514823 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6e410-9c30-42d5-8f28-d583e8ab9a38 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:49.516997 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 19/35] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:12:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:12:49.521706 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a6e410-9c30-42d5-8f28-d583e8ab9a38 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:12:49.590705 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f11203-f412-4385-8dde-ce58fa40c369 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:12:49.630404 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa955b6d-f45c-448f-868c-e7082a0331e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:12:49.631327 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa955b6d-f45c-448f-868c-e7082a0331e7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:49.631747 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa955b6d-f45c-448f-868c-e7082a0331e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:49.632149 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa955b6d-f45c-448f-868c-e7082a0331e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:49.632518 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a6e410-9c30-42d5-8f28-d583e8ab9a38 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:12:49.644286 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa955b6d-f45c-448f-868c-e7082a0331e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:12:49.645788 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa955b6d-f45c-448f-868c-e7082a0331e7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:12:49.652710 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:12:49.661296 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-aa955b6d-f45c-448f-868c-e7082a0331e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:12:49.661296 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:12:49.661296 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:12:49.661296 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:12:49.661296 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:12:49.661296 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:12:49.661296 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:12:49.661296 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:12:49.661296 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:12:49.661296 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:12:49.661296 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:12:49.661296 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:12:49.661296 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:12:49.661296 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:12:49.661296 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:12:49.662109 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:12:49.662109 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:12:49.662109 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:12:49.662109 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:12:49.662109 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:12:49.662109 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:12:49.662109 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:12:49.662109 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:12:49.662109 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:12:49.662109 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:12:49.662109 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:12:49.662109 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:12:49.662109 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:12:49.662109 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:12:49.662109 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:12:49.662109 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:12:49.662109 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:12:49.662935 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:12:49.662935 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:12:49.662935 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:12:49.662935 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:12:49.662935 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 17/36] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:12:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:12:49.663773 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ca7ac-a5ed-4c17-9ee0-1d4b3169e6a7 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:49.664059 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ca7ac-a5ed-4c17-9ee0-1d4b3169e6a7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:49.664332 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ca7ac-a5ed-4c17-9ee0-1d4b3169e6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:49.673347 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a4e2e6-ee3d-4035-925f-1fd345afda24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:12:49.674004 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4e2e6-ee3d-4035-925f-1fd345afda24 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:49.674287 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4e2e6-ee3d-4035-925f-1fd345afda24 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:49.674562 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4e2e6-ee3d-4035-925f-1fd345afda24 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:49.675871 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a4e2e6-ee3d-4035-925f-1fd345afda24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:12:49.677334 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a4e2e6-ee3d-4035-925f-1fd345afda24 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:12:49.679116 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 20/37] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:12:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:12:49.680169 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ada56-7a4f-4441-b54a-7aa6be69ff2c None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:49.680456 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ada56-7a4f-4441-b54a-7aa6be69ff2c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:49.680734 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ada56-7a4f-4441-b54a-7aa6be69ff2c None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:49.682041 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 21/38] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:12:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:12:49.682135 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e03ca7ac-a5ed-4c17-9ee0-1d4b3169e6a7 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:12:49.682671 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ada56-7a4f-4441-b54a-7aa6be69ff2c None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:12:49.684204 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e03ca7ac-a5ed-4c17-9ee0-1d4b3169e6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:12:49.698794 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:12:49.752538 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 18/39] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:12:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:12:49.759905 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ca7ac-a5ed-4c17-9ee0-1d4b3169e6a7 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:12:49.788007 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ada56-7a4f-4441-b54a-7aa6be69ff2c None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:12:49.822241 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75913cf1-fe18-4563-879a-91ffd85b1d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:12:49.822866 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75913cf1-fe18-4563-879a-91ffd85b1d00 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:49.823181 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75913cf1-fe18-4563-879a-91ffd85b1d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:49.823454 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75913cf1-fe18-4563-879a-91ffd85b1d00 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:49.824151 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75913cf1-fe18-4563-879a-91ffd85b1d00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:12:49.824453 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75913cf1-fe18-4563-879a-91ffd85b1d00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:12:49.825847 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75913cf1-fe18-4563-879a-91ffd85b1d00 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:12:49.847058 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 22/40] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:12:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:12:49.848433 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447b484-ba38-4775-b363-cc5ccbef7ce7 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:49.848744 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447b484-ba38-4775-b363-cc5ccbef7ce7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:49.849024 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447b484-ba38-4775-b363-cc5ccbef7ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:49.860255 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ca7ac-a5ed-4c17-9ee0-1d4b3169e6a7 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:12:49.867400 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-d447b484-ba38-4775-b363-cc5ccbef7ce7 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:12:49.868921 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d447b484-ba38-4775-b363-cc5ccbef7ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:12:49.886690 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:12:49.895404 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1192ec8-07ba-48a2-beb7-c0e5e169ff1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:12:49.896096 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1192ec8-07ba-48a2-beb7-c0e5e169ff1b None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:49.896380 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1192ec8-07ba-48a2-beb7-c0e5e169ff1b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:49.896664 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1192ec8-07ba-48a2-beb7-c0e5e169ff1b None admin] PATH_INFO: `/v3/domains/default/users/898d0645ae1e477983200e39184fa9ed/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:49.898702 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1192ec8-07ba-48a2-beb7-c0e5e169ff1b None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=898d0645ae1e477983200e39184fa9ed, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:12:49.900212 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1192ec8-07ba-48a2-beb7-c0e5e169ff1b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:12:49.916326 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 19/41] 10.222.0.40 () {58 vars in 1361 bytes} [Fri Aug 5 10:12:49 2022] PUT /identity/v3/domains/default/users/898d0645ae1e477983200e39184fa9ed/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:12:49.917137 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1192ec8-07ba-48a2-beb7-c0e5e169ff1b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:12:49.955169 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 23/42] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:12:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:12:49.956130 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447b484-ba38-4775-b363-cc5ccbef7ce7 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:12:49.958969 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:12:50.035029 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1192ec8-07ba-48a2-beb7-c0e5e169ff1b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:12:50.071393 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447b484-ba38-4775-b363-cc5ccbef7ce7 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:12:50.073740 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae0e98-dfff-4851-8c9c-979a5c2f6595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:12:50.074466 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae0e98-dfff-4851-8c9c-979a5c2f6595 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:50.074829 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae0e98-dfff-4851-8c9c-979a5c2f6595 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:50.075144 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae0e98-dfff-4851-8c9c-979a5c2f6595 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:50.075882 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bae0e98-dfff-4851-8c9c-979a5c2f6595 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:12:50.077315 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bae0e98-dfff-4851-8c9c-979a5c2f6595 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:12:50.094433 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 20/43] 10.222.0.40 () {60 vars in 1131 bytes} [Fri Aug 5 10:12:49 2022] POST /identity/v3/roles => generated 223 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:12:50.099999 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae0e98-dfff-4851-8c9c-979a5c2f6595 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:12:50.116576 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fef80a-60be-4840-8378-60ce75000adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:12:50.117281 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fef80a-60be-4840-8378-60ce75000adc None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:50.117559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fef80a-60be-4840-8378-60ce75000adc None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:50.117837 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fef80a-60be-4840-8378-60ce75000adc None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:50.118551 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fef80a-60be-4840-8378-60ce75000adc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:12:50.120179 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fef80a-60be-4840-8378-60ce75000adc None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:12:50.162169 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 24/44] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 5 10:12:49 2022] POST /identity/v3/projects => generated 301 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:12:50.183404 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:12:50.251896 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae0e98-dfff-4851-8c9c-979a5c2f6595 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:12:50.297006 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a57d92f-47a8-467e-93e2-825195439e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:12:50.297756 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57d92f-47a8-467e-93e2-825195439e4c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:50.298043 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57d92f-47a8-467e-93e2-825195439e4c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:50.298321 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57d92f-47a8-467e-93e2-825195439e4c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:50.299595 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a57d92f-47a8-467e-93e2-825195439e4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:12:50.301096 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a57d92f-47a8-467e-93e2-825195439e4c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:12:50.302983 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 21/45] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:12:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:12:50.309478 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fef80a-60be-4840-8378-60ce75000adc None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:12:50.371165 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:12:50.419294 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fef80a-60be-4840-8378-60ce75000adc None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:12:50.453858 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f7f8be-b9cc-40d1-a757-1bd074716198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:12:50.454525 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7f8be-b9cc-40d1-a757-1bd074716198 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:50.454813 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7f8be-b9cc-40d1-a757-1bd074716198 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:50.455120 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7f8be-b9cc-40d1-a757-1bd074716198 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:50.455834 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f7f8be-b9cc-40d1-a757-1bd074716198 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:12:50.457299 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f7f8be-b9cc-40d1-a757-1bd074716198 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:12:50.486302 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 25/46] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 5 10:12:50 2022] POST /identity/v3/projects => generated 312 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:12:52.023799 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab45ef96-c329-4679-85b0-426d2f84eb05 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:52.024344 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab45ef96-c329-4679-85b0-426d2f84eb05 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:52.024344 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab45ef96-c329-4679-85b0-426d2f84eb05 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:52.025744 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 22/47] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:12:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:12:52.031299 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5341eaf-e0cd-4b99-8685-74348a69efa0 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:52.032090 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5341eaf-e0cd-4b99-8685-74348a69efa0 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:52.032090 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5341eaf-e0cd-4b99-8685-74348a69efa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:52.051847 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a5341eaf-e0cd-4b99-8685-74348a69efa0 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:12:52.053077 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a5341eaf-e0cd-4b99-8685-74348a69efa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:12:52.124812 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 26/48] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:12:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:12:52.147496 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab45ef96-c329-4679-85b0-426d2f84eb05 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:12:52.255888 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab45ef96-c329-4679-85b0-426d2f84eb05 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:12:52.272368 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:12:52.291895 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b8f2aa-9972-4272-9f01-905c62e5a240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:12:52.292474 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b8f2aa-9972-4272-9f01-905c62e5a240 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:52.292760 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b8f2aa-9972-4272-9f01-905c62e5a240 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:52.293037 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b8f2aa-9972-4272-9f01-905c62e5a240 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:52.306584 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b8f2aa-9972-4272-9f01-905c62e5a240 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:12:52.312756 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-61b8f2aa-9972-4272-9f01-905c62e5a240 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:12:52.312756 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:12:52.312756 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:12:52.312756 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:12:52.312756 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:12:52.312756 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:12:52.312756 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:12:52.312756 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:12:52.312756 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:12:52.312756 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:12:52.312756 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:12:52.312756 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:12:52.312756 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:12:52.312756 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:12:52.312756 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:12:52.312756 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:12:52.313854 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:12:52.313854 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:12:52.313854 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:12:52.313854 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:12:52.313854 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:12:52.313854 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:12:52.313854 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:12:52.313854 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:12:52.313854 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:12:52.313854 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:12:52.313854 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:12:52.313854 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:12:52.313854 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:12:52.313854 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:12:52.313854 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:12:52.313854 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:12:52.314725 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:12:52.314725 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:12:52.314725 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:12:52.314725 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:12:52.314725 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:12:52.314725 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:12:52.314725 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:12:52.314725 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:12:52.314725 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:12:52.314725 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:12:52.314725 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:12:52.314725 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:12:52.314725 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 23/49] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 10:12:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:12:52.320097 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5341eaf-e0cd-4b99-8685-74348a69efa0 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:12:52.357155 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:12:52.424667 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5341eaf-e0cd-4b99-8685-74348a69efa0 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:12:52.456396 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d2c4c-9240-4c88-a27c-056d79ee79af None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:52.456658 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d2c4c-9240-4c88-a27c-056d79ee79af None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:52.456944 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d2c4c-9240-4c88-a27c-056d79ee79af None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:52.458319 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 24/50] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:12:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:12:52.459271 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84918ccf-6ca0-43b5-84d2-2d6cc69e4a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:12:52.459967 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84918ccf-6ca0-43b5-84d2-2d6cc69e4a39 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:52.460254 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84918ccf-6ca0-43b5-84d2-2d6cc69e4a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:52.460530 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84918ccf-6ca0-43b5-84d2-2d6cc69e4a39 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:52.461253 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84918ccf-6ca0-43b5-84d2-2d6cc69e4a39 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:12:52.461558 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84918ccf-6ca0-43b5-84d2-2d6cc69e4a39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:12:52.463100 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84918ccf-6ca0-43b5-84d2-2d6cc69e4a39 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:12:52.463367 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d02595-cfbd-4400-b51a-3b5277e48e1a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:52.463655 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d02595-cfbd-4400-b51a-3b5277e48e1a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:52.463935 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d02595-cfbd-4400-b51a-3b5277e48e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:52.471736 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 27/51] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:12:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:12:52.477348 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84918ccf-6ca0-43b5-84d2-2d6cc69e4a39 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:12:52.482719 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-81d02595-cfbd-4400-b51a-3b5277e48e1a None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:12:52.484019 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-81d02595-cfbd-4400-b51a-3b5277e48e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:12:52.550408 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 25/52] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:12:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:12:52.572921 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d02595-cfbd-4400-b51a-3b5277e48e1a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:12:52.573401 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:12:52.578167 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84918ccf-6ca0-43b5-84d2-2d6cc69e4a39 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:12:52.615033 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8bce8d-066a-4d56-b023-3bbc09000b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:12:52.615689 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8bce8d-066a-4d56-b023-3bbc09000b17 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:52.615988 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8bce8d-066a-4d56-b023-3bbc09000b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:52.616262 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8bce8d-066a-4d56-b023-3bbc09000b17 None admin] PATH_INFO: `/v3/users/898d0645ae1e477983200e39184fa9ed` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:52.617484 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8bce8d-066a-4d56-b023-3bbc09000b17 None admin] RBAC: Authorizing `identity:get_user(user_id=898d0645ae1e477983200e39184fa9ed)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:12:52.618926 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8bce8d-066a-4d56-b023-3bbc09000b17 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:12:52.620578 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 28/53] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:12:52 2022] GET /identity/v3/users/898d0645ae1e477983200e39184fa9ed => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:12:52.627264 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8bce8d-066a-4d56-b023-3bbc09000b17 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:12:52.683816 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d02595-cfbd-4400-b51a-3b5277e48e1a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:12:52.691034 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:12:52.718714 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c079f78b-0df8-4615-ba9a-31b83310e4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:12:52.719408 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079f78b-0df8-4615-ba9a-31b83310e4cf None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:52.719687 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079f78b-0df8-4615-ba9a-31b83310e4cf None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:52.719955 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079f78b-0df8-4615-ba9a-31b83310e4cf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:52.721101 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c079f78b-0df8-4615-ba9a-31b83310e4cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:12:52.722595 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c079f78b-0df8-4615-ba9a-31b83310e4cf None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:12:52.724198 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 26/54] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:12:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:12:52.729978 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c079f78b-0df8-4615-ba9a-31b83310e4cf None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:12:52.731262 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8bce8d-066a-4d56-b023-3bbc09000b17 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:12:52.764221 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc10a4e4-7894-45c3-86e0-6ebc2696995a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:12:52.764864 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10a4e4-7894-45c3-86e0-6ebc2696995a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:52.765252 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10a4e4-7894-45c3-86e0-6ebc2696995a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:52.765515 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10a4e4-7894-45c3-86e0-6ebc2696995a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:52.766978 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc10a4e4-7894-45c3-86e0-6ebc2696995a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:12:52.768770 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc10a4e4-7894-45c3-86e0-6ebc2696995a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:12:52.770394 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 29/55] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:12:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:12:52.776030 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc10a4e4-7894-45c3-86e0-6ebc2696995a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:12:52.812748 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:12:52.832968 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c079f78b-0df8-4615-ba9a-31b83310e4cf None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:12:52.867245 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9143f2-7051-4d52-8186-ef5d0a6a02c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:12:52.867875 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9143f2-7051-4d52-8186-ef5d0a6a02c6 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:52.868153 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9143f2-7051-4d52-8186-ef5d0a6a02c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:52.868431 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9143f2-7051-4d52-8186-ef5d0a6a02c6 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:52.869104 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9143f2-7051-4d52-8186-ef5d0a6a02c6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:12:52.870547 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9143f2-7051-4d52-8186-ef5d0a6a02c6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:12:52.882639 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc10a4e4-7894-45c3-86e0-6ebc2696995a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:12:52.899643 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 27/56] 10.222.0.40 () {60 vars in 1140 bytes} [Fri Aug 5 10:12:52 2022] POST /identity/v3/projects => generated 298 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:12:52.921425 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49431abf-c3e6-44b9-bd86-2224a32c1ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:12:52.922078 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49431abf-c3e6-44b9-bd86-2224a32c1ef2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:52.922513 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49431abf-c3e6-44b9-bd86-2224a32c1ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:52.922796 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49431abf-c3e6-44b9-bd86-2224a32c1ef2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:52.923655 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49431abf-c3e6-44b9-bd86-2224a32c1ef2 None admin] RBAC: Adding query filter params (role.id=daef89adcaeb48a382a0bcb03340ccbb, scope.domain.id=default, user.id=898d0645ae1e477983200e39184fa9ed) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:12:52.923972 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49431abf-c3e6-44b9-bd86-2224a32c1ef2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:12:52.925494 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49431abf-c3e6-44b9-bd86-2224a32c1ef2 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:12:52.932814 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 30/57] 10.222.0.40 () {56 vars in 1322 bytes} [Fri Aug 5 10:12:52 2022] GET /identity/v3/role_assignments?user.id=898d0645ae1e477983200e39184fa9ed&scope.domain.id=default&role.id=daef89adcaeb48a382a0bcb03340ccbb => generated 530 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:12:54.945884 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe190f-9267-4c2c-9321-adf3f0dc526c None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:54.946714 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe190f-9267-4c2c-9321-adf3f0dc526c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:54.946714 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe190f-9267-4c2c-9321-adf3f0dc526c None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:54.949022 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 28/58] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:12:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:12:54.955213 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388ff61-5d04-4e37-a18a-837281064697 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:54.955630 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388ff61-5d04-4e37-a18a-837281064697 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:54.956099 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388ff61-5d04-4e37-a18a-837281064697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:54.984734 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-9388ff61-5d04-4e37-a18a-837281064697 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:12:54.985966 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9388ff61-5d04-4e37-a18a-837281064697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:12:55.064117 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 31/59] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:12:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 10:12:55.087856 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fe190f-9267-4c2c-9321-adf3f0dc526c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:12:55.120195 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:12:55.206425 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fe190f-9267-4c2c-9321-adf3f0dc526c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:12:55.241575 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dfe17a-1f13-4a79-8497-5876969b3a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:12:55.242208 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dfe17a-1f13-4a79-8497-5876969b3a43 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:55.242488 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dfe17a-1f13-4a79-8497-5876969b3a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:55.242759 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dfe17a-1f13-4a79-8497-5876969b3a43 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:55.243985 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dfe17a-1f13-4a79-8497-5876969b3a43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:12:55.245504 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dfe17a-1f13-4a79-8497-5876969b3a43 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:12:55.247111 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 29/60] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:12:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:12:55.254042 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388ff61-5d04-4e37-a18a-837281064697 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:12:55.284280 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:12:55.374005 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388ff61-5d04-4e37-a18a-837281064697 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:12:55.424910 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2941af1-c340-4101-ae1f-502f17ece15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:12:55.425540 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2941af1-c340-4101-ae1f-502f17ece15f None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:55.425827 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2941af1-c340-4101-ae1f-502f17ece15f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:55.426105 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2941af1-c340-4101-ae1f-502f17ece15f None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:55.426798 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2941af1-c340-4101-ae1f-502f17ece15f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:12:55.428320 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2941af1-c340-4101-ae1f-502f17ece15f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:12:55.460258 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 32/61] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 5 10:12:55 2022] POST /identity/v3/users => generated 275 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:12:58.602463 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd3a89-72a2-4243-a9bf-f9a1e6870aae None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:58.603456 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd3a89-72a2-4243-a9bf-f9a1e6870aae None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:58.603976 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd3a89-72a2-4243-a9bf-f9a1e6870aae None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:58.605804 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 30/62] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:12:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:12:58.611122 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d22b30-378a-428f-b143-95381f8e6c3e None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:58.611939 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d22b30-378a-428f-b143-95381f8e6c3e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:58.612430 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d22b30-378a-428f-b143-95381f8e6c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:58.639908 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b9d22b30-378a-428f-b143-95381f8e6c3e None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:12:58.641643 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b9d22b30-378a-428f-b143-95381f8e6c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:12:58.733539 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 33/63] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:12:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:12:58.770542 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fd3a89-72a2-4243-a9bf-f9a1e6870aae None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:12:58.832774 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bdc92c-3279-4fb0-a272-20be8dc5b52b None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:58.833431 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bdc92c-3279-4fb0-a272-20be8dc5b52b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:58.833917 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bdc92c-3279-4fb0-a272-20be8dc5b52b None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:58.835807 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 34/64] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:12:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:12:58.852337 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de92ec83-2e9b-41e9-91bd-fe4d569c74a7 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:58.852975 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de92ec83-2e9b-41e9-91bd-fe4d569c74a7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:58.853486 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de92ec83-2e9b-41e9-91bd-fe4d569c74a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:58.886551 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:12:58.900026 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fd3a89-72a2-4243-a9bf-f9a1e6870aae None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:12:58.909959 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-de92ec83-2e9b-41e9-91bd-fe4d569c74a7 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:12:58.911592 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-de92ec83-2e9b-41e9-91bd-fe4d569c74a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:12:58.925576 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:12:58.941767 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a6be9-5cfd-499a-adfc-ceefa2d8f62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:12:58.942643 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a6be9-5cfd-499a-adfc-ceefa2d8f62e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:58.943094 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a6be9-5cfd-499a-adfc-ceefa2d8f62e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:58.943504 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a6be9-5cfd-499a-adfc-ceefa2d8f62e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:58.955475 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3a6be9-5cfd-499a-adfc-ceefa2d8f62e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:12:58.961340 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-ba3a6be9-5cfd-499a-adfc-ceefa2d8f62e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 10:12:58.961340 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:12:58.961340 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:12:58.961340 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:12:58.961340 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:12:58.961340 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:12:58.961340 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:12:58.961340 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:12:58.961340 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:12:58.961340 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:12:58.961340 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:12:58.961340 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:12:58.961340 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:12:58.961340 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:12:58.961340 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:12:58.961340 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:12:58.962381 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:12:58.962381 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:12:58.962381 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:12:58.962381 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:12:58.962381 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:12:58.962381 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:12:58.962381 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:12:58.962381 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:12:58.962381 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:12:58.962381 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:12:58.962381 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:12:58.962381 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:12:58.962381 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:12:58.962381 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:12:58.962381 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:12:58.962381 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:12:58.963393 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:12:58.963393 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:12:58.963393 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:12:58.963393 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:12:58.963393 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:12:58.963393 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:12:58.963393 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:12:58.963393 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:12:58.963393 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:12:58.963393 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:12:58.963393 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 10:12:58.963393 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:12:58.965436 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 31/65] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:12:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:12:58.967007 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e6b85a-13ce-4520-b0bd-41be1610d9bf None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:58.967414 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e6b85a-13ce-4520-b0bd-41be1610d9bf None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:58.967891 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e6b85a-13ce-4520-b0bd-41be1610d9bf None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:58.969369 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 32/66] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:12:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:12:58.974651 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e6b85a-13ce-4520-b0bd-41be1610d9bf None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:12:58.995943 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 35/67] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:12:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 10:12:59.004977 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d6d05-cd4f-4c37-b5b6-f78c0eba613c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:59.005592 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d6d05-cd4f-4c37-b5b6-f78c0eba613c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:59.007039 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d6d05-cd4f-4c37-b5b6-f78c0eba613c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:59.035627 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-746d6d05-cd4f-4c37-b5b6-f78c0eba613c None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:12:59.036845 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-746d6d05-cd4f-4c37-b5b6-f78c0eba613c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:12:59.108428 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e6b85a-13ce-4520-b0bd-41be1610d9bf None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:12:59.113849 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 36/68] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:12:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:12:59.114515 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d6d05-cd4f-4c37-b5b6-f78c0eba613c None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:12:59.151547 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2996d98b-358c-487f-b517-45d5851737f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:12:59.152219 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2996d98b-358c-487f-b517-45d5851737f2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:59.154860 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2996d98b-358c-487f-b517-45d5851737f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:59.154860 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2996d98b-358c-487f-b517-45d5851737f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:59.154860 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2996d98b-358c-487f-b517-45d5851737f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:12:59.154860 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2996d98b-358c-487f-b517-45d5851737f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:12:59.155219 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2996d98b-358c-487f-b517-45d5851737f2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:12:59.164859 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 33/69] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 10:12:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:12:59.166019 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602da0f3-4c29-420b-a1dc-f0fac0f1eff0 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:59.166311 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602da0f3-4c29-420b-a1dc-f0fac0f1eff0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:59.166585 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602da0f3-4c29-420b-a1dc-f0fac0f1eff0 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:59.167982 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 34/70] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:12:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:12:59.168856 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c27164-8ca5-4391-84d4-1f2b3b30481c None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:59.169310 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c27164-8ca5-4391-84d4-1f2b3b30481c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:59.169483 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c27164-8ca5-4391-84d4-1f2b3b30481c None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:59.171219 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 35/71] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:12:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:12:59.171893 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c27164-8ca5-4391-84d4-1f2b3b30481c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:12:59.190289 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:12:59.198227 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:12:59.231223 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d6d05-cd4f-4c37-b5b6-f78c0eba613c None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:12:59.267187 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6e569-c972-4521-b3ab-acbf7da579c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:12:59.267842 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6e569-c972-4521-b3ab-acbf7da579c8 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:59.268125 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6e569-c972-4521-b3ab-acbf7da579c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:59.268398 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6e569-c972-4521-b3ab-acbf7da579c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:59.278418 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c6e569-c972-4521-b3ab-acbf7da579c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:12:59.279339 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c27164-8ca5-4391-84d4-1f2b3b30481c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:12:59.284069 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-82c6e569-c972-4521-b3ab-acbf7da579c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:12:59.284069 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:12:59.284069 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:12:59.284069 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:12:59.284069 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:12:59.284069 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:12:59.284069 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:12:59.284069 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:12:59.284069 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:12:59.284069 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:12:59.284069 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:12:59.284069 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:12:59.284069 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:12:59.284069 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:12:59.284069 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:12:59.284069 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:12:59.284904 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:12:59.284904 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:12:59.284904 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:12:59.284904 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:12:59.284904 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:12:59.284904 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:12:59.284904 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:12:59.284904 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:12:59.284904 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:12:59.284904 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:12:59.284904 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:12:59.284904 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:12:59.284904 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:12:59.284904 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:12:59.284904 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:12:59.284904 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:12:59.285649 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:12:59.285649 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:12:59.285649 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:12:59.285649 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:12:59.285649 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:12:59.285649 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:12:59.285649 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:12:59.285649 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:12:59.285649 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:12:59.285649 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:12:59.285649 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:12:59.285649 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:12:59.285649 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 37/72] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 10:12:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:12:59.286270 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6e569-c972-4521-b3ab-acbf7da579c8 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:12:59.319341 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae70f3-d6be-4c56-b4d0-5351840e133a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:12:59.319994 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae70f3-d6be-4c56-b4d0-5351840e133a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:59.320272 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae70f3-d6be-4c56-b4d0-5351840e133a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:59.320543 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae70f3-d6be-4c56-b4d0-5351840e133a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:59.330864 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ae70f3-d6be-4c56-b4d0-5351840e133a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:12:59.335779 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-14ae70f3-d6be-4c56-b4d0-5351840e133a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 10:12:59.335779 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:12:59.335779 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:12:59.335779 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:12:59.335779 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:12:59.335779 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:12:59.335779 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:12:59.335779 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:12:59.335779 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:12:59.335779 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:12:59.335779 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:12:59.335779 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:12:59.335779 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:12:59.335779 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:12:59.335779 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:12:59.335779 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:12:59.336600 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:12:59.336600 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:12:59.336600 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:12:59.336600 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:12:59.336600 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:12:59.336600 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:12:59.336600 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:12:59.336600 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:12:59.336600 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:12:59.336600 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:12:59.336600 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:12:59.336600 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:12:59.336600 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:12:59.336600 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:12:59.336600 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:12:59.336600 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:12:59.337350 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:12:59.337350 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:12:59.337350 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:12:59.337350 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:12:59.337350 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:12:59.337350 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:12:59.337350 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:12:59.337350 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:12:59.337350 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:12:59.337350 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:12:59.337350 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 10:12:59.337350 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:12:59.337350 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 36/73] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 5 10:12:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:12:59.338426 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d1b4d6-12aa-4243-b125-0f540b2e0c06 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:59.338426 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d1b4d6-12aa-4243-b125-0f540b2e0c06 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:59.338595 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d1b4d6-12aa-4243-b125-0f540b2e0c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:59.355568 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d3d1b4d6-12aa-4243-b125-0f540b2e0c06 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:12:59.356746 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d3d1b4d6-12aa-4243-b125-0f540b2e0c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:12:59.386417 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6e569-c972-4521-b3ab-acbf7da579c8 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:12:59.420007 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4918c6d-841a-406a-ab2f-62ccb15eb334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:12:59.420987 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4918c6d-841a-406a-ab2f-62ccb15eb334 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:59.421266 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4918c6d-841a-406a-ab2f-62ccb15eb334 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:59.421532 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4918c6d-841a-406a-ab2f-62ccb15eb334 None admin] PATH_INFO: `/v3/users/f720a2ce258149afacb65b2c45fa2173` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:59.423045 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 37/74] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:12:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:12:59.424350 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:12:59.424444 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc4332-9927-4357-bcfa-e1a20db2e8f6 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:59.424666 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc4332-9927-4357-bcfa-e1a20db2e8f6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:59.424948 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc4332-9927-4357-bcfa-e1a20db2e8f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:59.437334 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:12:59.443505 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6afc4332-9927-4357-bcfa-e1a20db2e8f6 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:12:59.445382 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6afc4332-9927-4357-bcfa-e1a20db2e8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:12:59.452130 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4918c6d-841a-406a-ab2f-62ccb15eb334 None admin] RBAC: Authorizing `identity:get_user(user_id=f720a2ce258149afacb65b2c45fa2173)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:12:59.453685 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4918c6d-841a-406a-ab2f-62ccb15eb334 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:12:59.455423 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 38/75] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:12:59 2022] GET /identity/v3/users/f720a2ce258149afacb65b2c45fa2173 => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:12:59.460414 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4918c6d-841a-406a-ab2f-62ccb15eb334 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:12:59.520583 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 38/76] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:12:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:12:59.524148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afc4332-9927-4357-bcfa-e1a20db2e8f6 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:12:59.562620 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4918c6d-841a-406a-ab2f-62ccb15eb334 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:12:59.595729 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1470eb0-8c12-435e-8eef-60f45a99ba1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:12:59.596358 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1470eb0-8c12-435e-8eef-60f45a99ba1e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:59.596643 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1470eb0-8c12-435e-8eef-60f45a99ba1e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:59.596922 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1470eb0-8c12-435e-8eef-60f45a99ba1e None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:59.597647 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1470eb0-8c12-435e-8eef-60f45a99ba1e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:12:59.597955 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1470eb0-8c12-435e-8eef-60f45a99ba1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:12:59.599433 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1470eb0-8c12-435e-8eef-60f45a99ba1e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:12:59.607580 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 39/77] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:12:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:12:59.608377 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1470eb0-8c12-435e-8eef-60f45a99ba1e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:12:59.624851 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afc4332-9927-4357-bcfa-e1a20db2e8f6 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:12:59.659291 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1bf776-0e34-4ae7-88c6-bad363db7bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:12:59.659870 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1bf776-0e34-4ae7-88c6-bad363db7bf3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:59.660153 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1bf776-0e34-4ae7-88c6-bad363db7bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:59.660417 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1bf776-0e34-4ae7-88c6-bad363db7bf3 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:59.661139 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1bf776-0e34-4ae7-88c6-bad363db7bf3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:12:59.661448 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1bf776-0e34-4ae7-88c6-bad363db7bf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:12:59.662471 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:12:59.662895 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1bf776-0e34-4ae7-88c6-bad363db7bf3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:12:59.665384 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:12:59.672628 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 39/78] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:12:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:12:59.678121 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1bf776-0e34-4ae7-88c6-bad363db7bf3 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:12:59.708288 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1470eb0-8c12-435e-8eef-60f45a99ba1e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:12:59.742667 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dba7fc-244b-4926-abe0-f4213c660c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:12:59.743801 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dba7fc-244b-4926-abe0-f4213c660c4c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:59.744084 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dba7fc-244b-4926-abe0-f4213c660c4c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:59.744359 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dba7fc-244b-4926-abe0-f4213c660c4c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:59.745508 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dba7fc-244b-4926-abe0-f4213c660c4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:12:59.746978 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dba7fc-244b-4926-abe0-f4213c660c4c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:12:59.748605 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 40/79] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:12:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:12:59.749354 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dba7fc-244b-4926-abe0-f4213c660c4c None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:12:59.776424 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1bf776-0e34-4ae7-88c6-bad363db7bf3 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:12:59.809852 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83501cd2-8e4d-4fdd-9283-9817f7160bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:12:59.810513 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83501cd2-8e4d-4fdd-9283-9817f7160bab None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:59.810794 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83501cd2-8e4d-4fdd-9283-9817f7160bab None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:59.811100 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83501cd2-8e4d-4fdd-9283-9817f7160bab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:59.847717 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dba7fc-244b-4926-abe0-f4213c660c4c None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:12:59.869527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:12:59.874058 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:12:59.877295 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83501cd2-8e4d-4fdd-9283-9817f7160bab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:12:59.879188 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83501cd2-8e4d-4fdd-9283-9817f7160bab None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:12:59.881821 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3057e656-919e-48e8-8b88-3f7268e44aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:12:59.882453 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057e656-919e-48e8-8b88-3f7268e44aeb None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:12:59.882734 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057e656-919e-48e8-8b88-3f7268e44aeb None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:12:59.883032 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057e656-919e-48e8-8b88-3f7268e44aeb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:12:59.892896 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3057e656-919e-48e8-8b88-3f7268e44aeb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:12:59.898214 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-3057e656-919e-48e8-8b88-3f7268e44aeb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 10:12:59.898214 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:12:59.898214 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:12:59.898214 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:12:59.898214 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:12:59.898214 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:12:59.898214 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:12:59.898214 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:12:59.898214 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:12:59.898214 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:12:59.898214 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:12:59.898214 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:12:59.898214 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:12:59.898214 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:12:59.898214 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:12:59.898214 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:12:59.899078 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:12:59.899078 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:12:59.899078 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:12:59.899078 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:12:59.899078 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:12:59.899078 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:12:59.899078 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:12:59.899078 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:12:59.899078 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:12:59.899078 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:12:59.899078 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:12:59.899078 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:12:59.899078 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:12:59.899078 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:12:59.899078 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:12:59.899078 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:12:59.899828 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:12:59.899828 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:12:59.899828 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:12:59.899828 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:12:59.899828 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:12:59.899828 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:12:59.899828 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:12:59.899828 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:12:59.899828 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:12:59.899828 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:12:59.899828 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 10:12:59.899828 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:12:59.899828 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 41/80] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:12:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:12:59.900449 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3057e656-919e-48e8-8b88-3f7268e44aeb None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:12:59.983064 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 40/81] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:12:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 10:12:59.983922 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83501cd2-8e4d-4fdd-9283-9817f7160bab None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:00.001253 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3057e656-919e-48e8-8b88-3f7268e44aeb None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:00.035426 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5222230d-243c-4310-8a69-cdbb1ae577e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:00.036105 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222230d-243c-4310-8a69-cdbb1ae577e1 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:00.036388 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222230d-243c-4310-8a69-cdbb1ae577e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:00.036662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222230d-243c-4310-8a69-cdbb1ae577e1 None admin] PATH_INFO: `/v3/users/898d0645ae1e477983200e39184fa9ed` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:00.037906 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5222230d-243c-4310-8a69-cdbb1ae577e1 None admin] RBAC: Authorizing `identity:get_user(user_id=898d0645ae1e477983200e39184fa9ed)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:00.039566 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5222230d-243c-4310-8a69-cdbb1ae577e1 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:00.041194 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 42/82] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:12:59 2022] GET /identity/v3/users/898d0645ae1e477983200e39184fa9ed => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:00.042019 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5222230d-243c-4310-8a69-cdbb1ae577e1 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:00.079443 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:00.083501 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83501cd2-8e4d-4fdd-9283-9817f7160bab None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:00.090187 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:00.117197 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c533be-4d31-4708-8524-eb0bfffb6f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:00.117858 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c533be-4d31-4708-8524-eb0bfffb6f83 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:00.118142 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c533be-4d31-4708-8524-eb0bfffb6f83 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:00.118416 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c533be-4d31-4708-8524-eb0bfffb6f83 None admin] PATH_INFO: `/v3/users/f720a2ce258149afacb65b2c45fa2173` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:00.119654 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c533be-4d31-4708-8524-eb0bfffb6f83 None admin] RBAC: Authorizing `identity:get_user(user_id=f720a2ce258149afacb65b2c45fa2173)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:00.121128 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c533be-4d31-4708-8524-eb0bfffb6f83 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:00.122753 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 41/83] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:12:59 2022] GET /identity/v3/users/f720a2ce258149afacb65b2c45fa2173 => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:00.123551 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c533be-4d31-4708-8524-eb0bfffb6f83 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:00.139591 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5222230d-243c-4310-8a69-cdbb1ae577e1 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:00.173805 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7e34f-3d73-4e93-a84d-7362b14d5840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:00.174434 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7e34f-3d73-4e93-a84d-7362b14d5840 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:00.174714 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7e34f-3d73-4e93-a84d-7362b14d5840 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:00.175018 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7e34f-3d73-4e93-a84d-7362b14d5840 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:00.175714 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d7e34f-3d73-4e93-a84d-7362b14d5840 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:00.177181 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d7e34f-3d73-4e93-a84d-7362b14d5840 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:00.198452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 43/84] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 5 10:13:00 2022] POST /identity/v3/users => generated 289 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:13:00.199313 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7e34f-3d73-4e93-a84d-7362b14d5840 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:00.225977 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c533be-4d31-4708-8524-eb0bfffb6f83 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:00.268944 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65da5e-6dc8-42a9-bc2f-66efc32b002a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:00.269510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65da5e-6dc8-42a9-bc2f-66efc32b002a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:00.269791 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65da5e-6dc8-42a9-bc2f-66efc32b002a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:00.270065 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65da5e-6dc8-42a9-bc2f-66efc32b002a None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:00.270795 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c65da5e-6dc8-42a9-bc2f-66efc32b002a None admin] RBAC: Adding query filter params (name=member) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:00.271150 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c65da5e-6dc8-42a9-bc2f-66efc32b002a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:00.272574 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c65da5e-6dc8-42a9-bc2f-66efc32b002a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:00.283759 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 42/85] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 10:13:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:00.284447 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65da5e-6dc8-42a9-bc2f-66efc32b002a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:00.311185 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7e34f-3d73-4e93-a84d-7362b14d5840 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:00.319830 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:00.321478 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:00.352520 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48019679-ef9b-4f0f-be54-22174179a970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:00.353207 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48019679-ef9b-4f0f-be54-22174179a970 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:00.353466 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48019679-ef9b-4f0f-be54-22174179a970 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:00.353763 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48019679-ef9b-4f0f-be54-22174179a970 None admin] PATH_INFO: `/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:00.354978 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48019679-ef9b-4f0f-be54-22174179a970 None admin] RBAC: Authorizing `identity:get_project(project_id=a9d326f0c0fa4a38aa7ba8c8c6797c5a)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:00.356454 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48019679-ef9b-4f0f-be54-22174179a970 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:00.362712 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 44/86] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:13:00 2022] GET /identity/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:00.362712 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48019679-ef9b-4f0f-be54-22174179a970 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:00.424403 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65da5e-6dc8-42a9-bc2f-66efc32b002a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:00.473838 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf20c7e-51f9-43e8-8f8f-fba03b6a7f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:00.474487 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf20c7e-51f9-43e8-8f8f-fba03b6a7f52 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:00.474892 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf20c7e-51f9-43e8-8f8f-fba03b6a7f52 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:00.475201 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf20c7e-51f9-43e8-8f8f-fba03b6a7f52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:00.479118 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48019679-ef9b-4f0f-be54-22174179a970 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:00.516287 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600d3ea4-ecc4-4951-a228-204559d3676a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:00.516928 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600d3ea4-ecc4-4951-a228-204559d3676a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:00.517216 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600d3ea4-ecc4-4951-a228-204559d3676a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:00.517494 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600d3ea4-ecc4-4951-a228-204559d3676a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:00.559850 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf20c7e-51f9-43e8-8f8f-fba03b6a7f52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:00.561277 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf20c7e-51f9-43e8-8f8f-fba03b6a7f52 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:00.583338 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:00.590454 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:00.608247 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600d3ea4-ecc4-4951-a228-204559d3676a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:00.610643 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600d3ea4-ecc4-4951-a228-204559d3676a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:00.671546 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 43/87] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:13:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 10:13:00.673083 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf20c7e-51f9-43e8-8f8f-fba03b6a7f52 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:00.757097 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 45/88] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:13:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 10:13:00.759265 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600d3ea4-ecc4-4951-a228-204559d3676a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:00.782153 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf20c7e-51f9-43e8-8f8f-fba03b6a7f52 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:00.810029 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:00.820479 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf30db00-0f9b-406b-95b6-9eb124f64244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:00.821336 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf30db00-0f9b-406b-95b6-9eb124f64244 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:00.821737 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf30db00-0f9b-406b-95b6-9eb124f64244 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:00.822141 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf30db00-0f9b-406b-95b6-9eb124f64244 None admin] PATH_INFO: `/v3/users/f720a2ce258149afacb65b2c45fa2173` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:00.823457 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf30db00-0f9b-406b-95b6-9eb124f64244 None admin] RBAC: Authorizing `identity:get_user(user_id=f720a2ce258149afacb65b2c45fa2173)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:00.825031 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf30db00-0f9b-406b-95b6-9eb124f64244 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:00.826771 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 44/89] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:13:00 2022] GET /identity/v3/users/f720a2ce258149afacb65b2c45fa2173 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:00.832485 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf30db00-0f9b-406b-95b6-9eb124f64244 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:00.871310 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:00.883814 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600d3ea4-ecc4-4951-a228-204559d3676a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:00.926533 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc982c0-0e7b-4ac2-895d-4a5edbc48e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:00.927651 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc982c0-0e7b-4ac2-895d-4a5edbc48e8b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:00.928034 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc982c0-0e7b-4ac2-895d-4a5edbc48e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:00.928408 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc982c0-0e7b-4ac2-895d-4a5edbc48e8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:00.929477 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc982c0-0e7b-4ac2-895d-4a5edbc48e8b None admin] RBAC: Adding query filter params (role.id=c4ba94fa4bd34a09ba87d357142d9db8, scope.project.id=a9d326f0c0fa4a38aa7ba8c8c6797c5a, user.id=f720a2ce258149afacb65b2c45fa2173) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:00.929891 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc982c0-0e7b-4ac2-895d-4a5edbc48e8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:00.931814 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc982c0-0e7b-4ac2-895d-4a5edbc48e8b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:00.939357 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 46/90] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:13:00 2022] GET /identity/v3/role_assignments?user.id=f720a2ce258149afacb65b2c45fa2173&scope.project.id=a9d326f0c0fa4a38aa7ba8c8c6797c5a&role.id=c4ba94fa4bd34a09ba87d357142d9db8 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:00.943846 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc982c0-0e7b-4ac2-895d-4a5edbc48e8b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:00.960900 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf30db00-0f9b-406b-95b6-9eb124f64244 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:01.039208 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e12e3-981f-4b7f-934e-28587d3c6d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:01.041484 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e12e3-981f-4b7f-934e-28587d3c6d6c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:01.041962 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e12e3-981f-4b7f-934e-28587d3c6d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:01.042373 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e12e3-981f-4b7f-934e-28587d3c6d6c None admin] PATH_INFO: `/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:01.045864 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073e12e3-981f-4b7f-934e-28587d3c6d6c None admin] RBAC: Authorizing `identity:get_project(project_id=a9d326f0c0fa4a38aa7ba8c8c6797c5a)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:01.051213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073e12e3-981f-4b7f-934e-28587d3c6d6c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:01.056381 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 45/91] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:13:00 2022] GET /identity/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:01.057388 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e12e3-981f-4b7f-934e-28587d3c6d6c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:01.152716 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc982c0-0e7b-4ac2-895d-4a5edbc48e8b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:01.168871 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:01.193072 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dc22da-e8ca-4e0f-bd6d-399e3f18b1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:01.194089 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc22da-e8ca-4e0f-bd6d-399e3f18b1b9 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:01.194558 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc22da-e8ca-4e0f-bd6d-399e3f18b1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:01.195129 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc22da-e8ca-4e0f-bd6d-399e3f18b1b9 None admin] PATH_INFO: `/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:01.196477 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dc22da-e8ca-4e0f-bd6d-399e3f18b1b9 None admin] RBAC: Authorizing `identity:get_project(project_id=a9d326f0c0fa4a38aa7ba8c8c6797c5a)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:01.198204 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dc22da-e8ca-4e0f-bd6d-399e3f18b1b9 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:01.200101 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 47/92] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:13:00 2022] GET /identity/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a => generated 298 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:01.201122 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dc22da-e8ca-4e0f-bd6d-399e3f18b1b9 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:01.210934 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e12e3-981f-4b7f-934e-28587d3c6d6c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:01.214886 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:01.254355 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c328ed-b58e-4496-90a0-011a435d640a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:01.256128 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c328ed-b58e-4496-90a0-011a435d640a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:01.257282 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c328ed-b58e-4496-90a0-011a435d640a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:01.257941 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c328ed-b58e-4496-90a0-011a435d640a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:01.332226 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c328ed-b58e-4496-90a0-011a435d640a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:01.334073 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c328ed-b58e-4496-90a0-011a435d640a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:01.383964 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dc22da-e8ca-4e0f-bd6d-399e3f18b1b9 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:01.433704 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:01.445337 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af076b5-3b3d-4057-b6c1-035e70f2d1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:01.446044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af076b5-3b3d-4057-b6c1-035e70f2d1e5 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:01.446327 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af076b5-3b3d-4057-b6c1-035e70f2d1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:01.446850 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af076b5-3b3d-4057-b6c1-035e70f2d1e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:01.449389 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af076b5-3b3d-4057-b6c1-035e70f2d1e5 None admin] RBAC: Adding query filter params (role.id=daef89adcaeb48a382a0bcb03340ccbb, scope.project.id=a9d326f0c0fa4a38aa7ba8c8c6797c5a, user.id=898d0645ae1e477983200e39184fa9ed) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:01.449712 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af076b5-3b3d-4057-b6c1-035e70f2d1e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:01.451290 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af076b5-3b3d-4057-b6c1-035e70f2d1e5 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:01.466407 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 48/93] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:13:01 2022] GET /identity/v3/role_assignments?user.id=898d0645ae1e477983200e39184fa9ed&scope.project.id=a9d326f0c0fa4a38aa7ba8c8c6797c5a&role.id=daef89adcaeb48a382a0bcb03340ccbb => generated 259 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:01.467302 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af076b5-3b3d-4057-b6c1-035e70f2d1e5 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:01.530638 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 46/94] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:13:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 10:13:01.539380 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c328ed-b58e-4496-90a0-011a435d640a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:01.654389 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:01.710535 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af076b5-3b3d-4057-b6c1-035e70f2d1e5 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:01.773082 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c328ed-b58e-4496-90a0-011a435d640a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:01.780685 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be6ab52-06b3-44cc-ba1c-abb4fec40a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:01.781394 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be6ab52-06b3-44cc-ba1c-abb4fec40a34 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:01.781624 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be6ab52-06b3-44cc-ba1c-abb4fec40a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:01.781839 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be6ab52-06b3-44cc-ba1c-abb4fec40a34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:01.782655 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be6ab52-06b3-44cc-ba1c-abb4fec40a34 None admin] RBAC: Adding query filter params (role.id=57a070b43ad940f1886c69721dd304c2, scope.project.id=a9d326f0c0fa4a38aa7ba8c8c6797c5a, user.id=f720a2ce258149afacb65b2c45fa2173) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:01.782992 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be6ab52-06b3-44cc-ba1c-abb4fec40a34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:01.784477 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be6ab52-06b3-44cc-ba1c-abb4fec40a34 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:01.796370 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 49/95] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:13:01 2022] GET /identity/v3/role_assignments?user.id=f720a2ce258149afacb65b2c45fa2173&scope.project.id=a9d326f0c0fa4a38aa7ba8c8c6797c5a&role.id=57a070b43ad940f1886c69721dd304c2 => generated 259 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:01.857254 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4674d8e5-dfd9-40d7-989d-0bdcee000811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:01.857919 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4674d8e5-dfd9-40d7-989d-0bdcee000811 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:01.858207 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4674d8e5-dfd9-40d7-989d-0bdcee000811 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:01.858488 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4674d8e5-dfd9-40d7-989d-0bdcee000811 None admin] PATH_INFO: `/v3/projects/387614ebbd51447b89e7a0c2ee6d4b96` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:01.860125 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4674d8e5-dfd9-40d7-989d-0bdcee000811 None admin] RBAC: Authorizing `identity:get_project(project_id=387614ebbd51447b89e7a0c2ee6d4b96)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:01.861590 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4674d8e5-dfd9-40d7-989d-0bdcee000811 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:01.863334 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 47/96] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:13:01 2022] GET /identity/v3/projects/387614ebbd51447b89e7a0c2ee6d4b96 => generated 312 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:01.869621 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be6ab52-06b3-44cc-ba1c-abb4fec40a34 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:02.019825 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be6ab52-06b3-44cc-ba1c-abb4fec40a34 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:02.067009 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9a5d9c-9ae1-4290-b645-0455e6635d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:02.068111 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a5d9c-9ae1-4290-b645-0455e6635d9d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:02.068608 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a5d9c-9ae1-4290-b645-0455e6635d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:02.069088 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a5d9c-9ae1-4290-b645-0455e6635d9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:02.070097 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9a5d9c-9ae1-4290-b645-0455e6635d9d None admin] RBAC: Adding query filter params (role.id=c4ba94fa4bd34a09ba87d357142d9db8, scope.project.id=387614ebbd51447b89e7a0c2ee6d4b96, user.id=f720a2ce258149afacb65b2c45fa2173) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:02.070634 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9a5d9c-9ae1-4290-b645-0455e6635d9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:02.076575 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9a5d9c-9ae1-4290-b645-0455e6635d9d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:02.079205 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:02.089520 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 50/97] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:13:01 2022] GET /identity/v3/role_assignments?user.id=f720a2ce258149afacb65b2c45fa2173&scope.project.id=387614ebbd51447b89e7a0c2ee6d4b96&role.id=c4ba94fa4bd34a09ba87d357142d9db8 => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:02.610636 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be33f07-2f6d-413d-8bc4-5991c508eccd None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:02.611032 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be33f07-2f6d-413d-8bc4-5991c508eccd None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:02.611227 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be33f07-2f6d-413d-8bc4-5991c508eccd None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:02.612616 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 48/98] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:02.617931 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d014e27-5ffb-49df-beb2-7231bce7b16f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:02.618221 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d014e27-5ffb-49df-beb2-7231bce7b16f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:02.618501 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d014e27-5ffb-49df-beb2-7231bce7b16f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:02.639110 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-2d014e27-5ffb-49df-beb2-7231bce7b16f None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:02.639110 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2d014e27-5ffb-49df-beb2-7231bce7b16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:02.760332 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 51/99] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:02.781535 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab612ae-4b7f-4666-8536-949b40d9f2f9 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:02.782100 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab612ae-4b7f-4666-8536-949b40d9f2f9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:02.782320 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab612ae-4b7f-4666-8536-949b40d9f2f9 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:02.783773 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 49/100] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:02.791791 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d014e27-5ffb-49df-beb2-7231bce7b16f None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:02.798779 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587149b-413a-4c4c-9172-e7526149c4f3 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:02.799149 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587149b-413a-4c4c-9172-e7526149c4f3 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:02.799380 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587149b-413a-4c4c-9172-e7526149c4f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:02.825747 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8587149b-413a-4c4c-9172-e7526149c4f3 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:02.826997 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8587149b-413a-4c4c-9172-e7526149c4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:02.896105 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:02.995454 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d014e27-5ffb-49df-beb2-7231bce7b16f None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:03.000398 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 50/101] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 10:13:03.033740 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:03.047741 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9141efc-e654-4b0f-ba44-7f8a8699b7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:03.048412 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9141efc-e654-4b0f-ba44-7f8a8699b7ff None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:03.048693 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9141efc-e654-4b0f-ba44-7f8a8699b7ff None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:03.048965 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9141efc-e654-4b0f-ba44-7f8a8699b7ff None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:03.062673 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9141efc-e654-4b0f-ba44-7f8a8699b7ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:03.067862 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-f9141efc-e654-4b0f-ba44-7f8a8699b7ff None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 10:13:03.067862 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:03.067862 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:03.067862 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:03.067862 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:03.067862 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:03.067862 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:03.067862 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:03.067862 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:03.067862 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:03.067862 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:03.067862 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:03.067862 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:03.067862 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:03.067862 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:03.067862 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:03.068699 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:03.068699 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:03.068699 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:03.068699 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:03.068699 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:03.068699 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:03.068699 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:03.068699 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:03.068699 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:03.068699 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:03.068699 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:03.068699 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:03.068699 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:03.068699 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:03.068699 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:03.068699 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:03.069444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:03.069444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:03.069444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:03.069444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:03.069444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:03.069444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:03.069444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:03.069444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:03.069444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:03.069444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:03.069444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 10:13:03.069444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:13:03.069444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 52/102] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:13:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:03.071325 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8587149b-413a-4c4c-9172-e7526149c4f3 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:03.076753 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9141efc-e654-4b0f-ba44-7f8a8699b7ff None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:03.218642 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9141efc-e654-4b0f-ba44-7f8a8699b7ff None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:03.251578 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8587149b-413a-4c4c-9172-e7526149c4f3 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:03.287426 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eced6c4-aef1-49b6-a724-930ec0940c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:03.288092 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eced6c4-aef1-49b6-a724-930ec0940c24 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:03.288384 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eced6c4-aef1-49b6-a724-930ec0940c24 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:03.288663 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eced6c4-aef1-49b6-a724-930ec0940c24 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:03.289833 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eced6c4-aef1-49b6-a724-930ec0940c24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:03.291349 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eced6c4-aef1-49b6-a724-930ec0940c24 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:03.291891 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61372a8a-687d-4d07-8cd4-0f819ddc68b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:03.292552 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61372a8a-687d-4d07-8cd4-0f819ddc68b4 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:03.292818 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61372a8a-687d-4d07-8cd4-0f819ddc68b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:03.292989 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 51/103] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:13:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:03.293079 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61372a8a-687d-4d07-8cd4-0f819ddc68b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:03.293815 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61372a8a-687d-4d07-8cd4-0f819ddc68b4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:03.294124 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61372a8a-687d-4d07-8cd4-0f819ddc68b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:03.298925 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61372a8a-687d-4d07-8cd4-0f819ddc68b4 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:03.302751 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eced6c4-aef1-49b6-a724-930ec0940c24 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:03.309863 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 53/104] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 10:13:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:13:03.321281 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61372a8a-687d-4d07-8cd4-0f819ddc68b4 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:03.341050 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:03.415038 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eced6c4-aef1-49b6-a724-930ec0940c24 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:03.423985 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:03.452080 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92accacf-27b4-4258-bf52-8ca45ca1da3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:03.453166 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92accacf-27b4-4258-bf52-8ca45ca1da3a None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:03.453647 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92accacf-27b4-4258-bf52-8ca45ca1da3a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:03.454120 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92accacf-27b4-4258-bf52-8ca45ca1da3a None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:03.455031 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92accacf-27b4-4258-bf52-8ca45ca1da3a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:03.456689 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92accacf-27b4-4258-bf52-8ca45ca1da3a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:03.489856 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 52/105] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 5 10:13:03 2022] POST /identity/v3/projects => generated 302 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:13:03.521477 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61372a8a-687d-4d07-8cd4-0f819ddc68b4 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:03.567348 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a7b8a6-efa9-4649-a189-8136f347c2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:03.568434 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a7b8a6-efa9-4649-a189-8136f347c2f2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:03.569038 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a7b8a6-efa9-4649-a189-8136f347c2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:03.569520 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a7b8a6-efa9-4649-a189-8136f347c2f2 None admin] PATH_INFO: `/v3/users/dd861837dc1d4bb2a5bcc19c545b8432` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:03.590370 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a7b8a6-efa9-4649-a189-8136f347c2f2 None admin] RBAC: Authorizing `identity:get_user(user_id=dd861837dc1d4bb2a5bcc19c545b8432)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:03.592310 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a7b8a6-efa9-4649-a189-8136f347c2f2 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:03.594427 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 54/106] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:13:03 2022] GET /identity/v3/users/dd861837dc1d4bb2a5bcc19c545b8432 => generated 289 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:13:03.600216 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92accacf-27b4-4258-bf52-8ca45ca1da3a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:03.710774 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:03.732480 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92accacf-27b4-4258-bf52-8ca45ca1da3a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:03.775597 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03c6f2-9360-45b1-8aa1-c0f118452e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:03.776984 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03c6f2-9360-45b1-8aa1-c0f118452e84 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:03.777678 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03c6f2-9360-45b1-8aa1-c0f118452e84 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:03.778343 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03c6f2-9360-45b1-8aa1-c0f118452e84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:03.855901 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a03c6f2-9360-45b1-8aa1-c0f118452e84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:03.858271 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a03c6f2-9360-45b1-8aa1-c0f118452e84 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:03.944188 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:03.963607 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 53/107] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:13:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 10:13:03.972072 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a7b8a6-efa9-4649-a189-8136f347c2f2 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:04.089816 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a7b8a6-efa9-4649-a189-8136f347c2f2 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:04.100675 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:04.127551 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e88bb-f2ee-481d-bccd-7ad4569c3da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:04.128266 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e88bb-f2ee-481d-bccd-7ad4569c3da9 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:04.128606 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e88bb-f2ee-481d-bccd-7ad4569c3da9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:04.128884 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e88bb-f2ee-481d-bccd-7ad4569c3da9 None admin] PATH_INFO: `/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:04.130102 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e88bb-f2ee-481d-bccd-7ad4569c3da9 None admin] RBAC: Authorizing `identity:get_project(project_id=a9d326f0c0fa4a38aa7ba8c8c6797c5a)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:04.131645 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e88bb-f2ee-481d-bccd-7ad4569c3da9 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:04.133379 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 55/108] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:13:03 2022] GET /identity/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:04.139496 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03c6f2-9360-45b1-8aa1-c0f118452e84 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:04.152206 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa5bc3-c6f4-4adc-bec1-c36304d488da None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:04.152206 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa5bc3-c6f4-4adc-bec1-c36304d488da None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:04.152440 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa5bc3-c6f4-4adc-bec1-c36304d488da None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:04.153676 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 56/109] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:04.164590 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4babde43-1c40-443e-94f9-8428e948f750 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:04.164813 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4babde43-1c40-443e-94f9-8428e948f750 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:04.165090 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4babde43-1c40-443e-94f9-8428e948f750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:04.199956 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-4babde43-1c40-443e-94f9-8428e948f750 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:04.201149 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4babde43-1c40-443e-94f9-8428e948f750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:04.321430 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 57/110] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 10:13:04.327126 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03c6f2-9360-45b1-8aa1-c0f118452e84 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:04.355511 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4babde43-1c40-443e-94f9-8428e948f750 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:04.379083 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca86a8a-ce51-44f3-a922-140b4937d6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:04.379726 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca86a8a-ce51-44f3-a922-140b4937d6bf None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:04.380004 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca86a8a-ce51-44f3-a922-140b4937d6bf None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:04.380277 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca86a8a-ce51-44f3-a922-140b4937d6bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:04.381620 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca86a8a-ce51-44f3-a922-140b4937d6bf None admin] RBAC: Adding query filter params (role.id=cf5de414a035475b8b75626cb19a2de2, scope.project.id=a9d326f0c0fa4a38aa7ba8c8c6797c5a, user.id=dd861837dc1d4bb2a5bcc19c545b8432) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:04.381933 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca86a8a-ce51-44f3-a922-140b4937d6bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:04.383951 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca86a8a-ce51-44f3-a922-140b4937d6bf None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:04.401966 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 54/111] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:13:04 2022] GET /identity/v3/role_assignments?user.id=dd861837dc1d4bb2a5bcc19c545b8432&scope.project.id=a9d326f0c0fa4a38aa7ba8c8c6797c5a&role.id=cf5de414a035475b8b75626cb19a2de2 => generated 259 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:04.405469 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e30f8fe-1d9d-4037-bcd2-6392f27094c7 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:04.405750 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e30f8fe-1d9d-4037-bcd2-6392f27094c7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:04.406302 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e30f8fe-1d9d-4037-bcd2-6392f27094c7 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:04.407583 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 55/112] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:04.416474 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f299679-f893-487e-8cc4-f306c7dee4c0 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:04.416783 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f299679-f893-487e-8cc4-f306c7dee4c0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:04.417033 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f299679-f893-487e-8cc4-f306c7dee4c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:04.448352 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-5f299679-f893-487e-8cc4-f306c7dee4c0 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:04.449578 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5f299679-f893-487e-8cc4-f306c7dee4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:04.533719 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4babde43-1c40-443e-94f9-8428e948f750 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:04.547170 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:04.558931 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:04.605532 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 56/113] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:04.611766 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8bc7db-a6ef-4688-88f0-a58ec71fe945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:04.612445 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8bc7db-a6ef-4688-88f0-a58ec71fe945 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:04.612727 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8bc7db-a6ef-4688-88f0-a58ec71fe945 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:04.613002 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8bc7db-a6ef-4688-88f0-a58ec71fe945 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:04.634330 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f299679-f893-487e-8cc4-f306c7dee4c0 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:04.636292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8bc7db-a6ef-4688-88f0-a58ec71fe945 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:04.641564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-9d8bc7db-a6ef-4688-88f0-a58ec71fe945 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:13:04.641564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:04.641564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:04.641564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:04.641564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:04.641564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:04.641564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:04.641564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:04.641564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:04.641564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:04.641564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:04.641564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:04.641564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:04.641564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:04.641564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:04.641564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:04.642409 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:04.642409 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:04.642409 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:04.642409 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:04.642409 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:04.642409 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:04.642409 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:04.642409 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:04.642409 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:04.642409 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:04.642409 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:04.642409 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:04.642409 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:04.642409 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:04.642409 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:04.642409 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:04.643190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:04.643190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:04.643190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:04.643190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:04.643190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:04.643190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:04.643190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:04.643190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:04.643190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:04.643190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:04.643190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:13:04.643190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:13:04.646120 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 58/114] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 10:13:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:04.655007 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8bc7db-a6ef-4688-88f0-a58ec71fe945 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:04.785541 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f299679-f893-487e-8cc4-f306c7dee4c0 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:04.838331 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b404d7-bac6-4ae8-a3e4-5166eadf6a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:04.840040 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b404d7-bac6-4ae8-a3e4-5166eadf6a55 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:04.840677 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b404d7-bac6-4ae8-a3e4-5166eadf6a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:04.842220 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b404d7-bac6-4ae8-a3e4-5166eadf6a55 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:04.861370 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8bc7db-a6ef-4688-88f0-a58ec71fe945 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:04.871888 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b404d7-bac6-4ae8-a3e4-5166eadf6a55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:04.879747 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-c9b404d7-bac6-4ae8-a3e4-5166eadf6a55 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 10:13:04.879747 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:04.879747 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:04.879747 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:04.879747 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:04.879747 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:04.879747 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:04.879747 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:04.879747 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:04.879747 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:04.879747 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:04.879747 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:04.879747 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:04.879747 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:04.879747 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:04.879747 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:04.881061 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:04.881061 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:04.881061 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:04.881061 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:04.881061 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:04.881061 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:04.881061 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:04.881061 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:04.881061 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:04.881061 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:04.881061 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:04.881061 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:04.881061 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:04.881061 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:04.881061 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:04.881061 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:04.882233 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:04.882233 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:04.882233 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:04.882233 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:04.882233 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:04.882233 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:04.882233 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:04.882233 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:04.882233 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:04.882233 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:04.882233 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 10:13:04.882233 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:04.885278 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 57/115] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:13:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:13:04.891653 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b404d7-bac6-4ae8-a3e4-5166eadf6a55 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:04.919400 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcf3387-7ac9-4f14-856b-67168b883cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:04.919961 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf3387-7ac9-4f14-856b-67168b883cde None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:04.920590 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf3387-7ac9-4f14-856b-67168b883cde None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:04.920863 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf3387-7ac9-4f14-856b-67168b883cde None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:04.921711 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcf3387-7ac9-4f14-856b-67168b883cde None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:04.926979 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcf3387-7ac9-4f14-856b-67168b883cde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:04.926979 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcf3387-7ac9-4f14-856b-67168b883cde None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:04.932168 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 59/116] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:13:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:13:04.937801 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:04.937943 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcf3387-7ac9-4f14-856b-67168b883cde None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:04.982734 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:05.035094 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b404d7-bac6-4ae8-a3e4-5166eadf6a55 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:05.113107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcf3387-7ac9-4f14-856b-67168b883cde None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:05.135638 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f445a-e9c0-422d-9079-5f8ead42a6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:05.136294 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f445a-e9c0-422d-9079-5f8ead42a6cb None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:05.136587 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f445a-e9c0-422d-9079-5f8ead42a6cb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:05.136870 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f445a-e9c0-422d-9079-5f8ead42a6cb None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:05.137620 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45f445a-e9c0-422d-9079-5f8ead42a6cb None admin] RBAC: Adding query filter params (name=member) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:05.137931 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45f445a-e9c0-422d-9079-5f8ead42a6cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:05.139388 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45f445a-e9c0-422d-9079-5f8ead42a6cb None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:05.152102 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 58/117] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 10:13:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:13:05.159406 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f445a-e9c0-422d-9079-5f8ead42a6cb None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:05.185788 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b77402-69d6-416f-936f-1dbd1df52c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:05.186497 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b77402-69d6-416f-936f-1dbd1df52c61 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:05.186780 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b77402-69d6-416f-936f-1dbd1df52c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:05.187081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b77402-69d6-416f-936f-1dbd1df52c61 None admin] PATH_INFO: `/v3/users/898d0645ae1e477983200e39184fa9ed` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:05.188875 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b77402-69d6-416f-936f-1dbd1df52c61 None admin] RBAC: Authorizing `identity:get_user(user_id=898d0645ae1e477983200e39184fa9ed)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:05.190929 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b77402-69d6-416f-936f-1dbd1df52c61 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:05.193362 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 60/118] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:13:04 2022] GET /identity/v3/users/898d0645ae1e477983200e39184fa9ed => generated 247 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:05.200008 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b77402-69d6-416f-936f-1dbd1df52c61 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:05.327504 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:05.333491 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f445a-e9c0-422d-9079-5f8ead42a6cb None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:05.340691 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:05.348208 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b77402-69d6-416f-936f-1dbd1df52c61 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:05.379478 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7016cb66-748b-4f85-8ea8-5072d71eca34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:05.380432 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7016cb66-748b-4f85-8ea8-5072d71eca34 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:05.380959 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7016cb66-748b-4f85-8ea8-5072d71eca34 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:05.381238 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7016cb66-748b-4f85-8ea8-5072d71eca34 None admin] PATH_INFO: `/v3/users/f720a2ce258149afacb65b2c45fa2173` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:05.382423 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7016cb66-748b-4f85-8ea8-5072d71eca34 None admin] RBAC: Authorizing `identity:get_user(user_id=f720a2ce258149afacb65b2c45fa2173)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:05.383916 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7016cb66-748b-4f85-8ea8-5072d71eca34 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:05.384872 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a04c7f0-b4d8-4187-9d0a-b525bd7a1978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:05.385525 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 59/119] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:13:05 2022] GET /identity/v3/users/f720a2ce258149afacb65b2c45fa2173 => generated 275 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:05.385644 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a04c7f0-b4d8-4187-9d0a-b525bd7a1978 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:05.386639 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1f6c6-c0aa-46d2-9de5-34caf2992d6d None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:05.386934 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1f6c6-c0aa-46d2-9de5-34caf2992d6d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:05.387217 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1f6c6-c0aa-46d2-9de5-34caf2992d6d None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:05.388395 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 60/120] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:05.389397 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922757d-fb0a-41b1-9e32-c87ce28314d6 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:05.389675 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922757d-fb0a-41b1-9e32-c87ce28314d6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:05.389949 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922757d-fb0a-41b1-9e32-c87ce28314d6 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:05.390812 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a04c7f0-b4d8-4187-9d0a-b525bd7a1978 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:05.391204 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a04c7f0-b4d8-4187-9d0a-b525bd7a1978 None admin] PATH_INFO: `/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:05.391782 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 61/121] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:05.392504 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922757d-fb0a-41b1-9e32-c87ce28314d6 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:05.392725 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a04c7f0-b4d8-4187-9d0a-b525bd7a1978 None admin] RBAC: Authorizing `identity:get_project(project_id=a9d326f0c0fa4a38aa7ba8c8c6797c5a)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:05.394393 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a04c7f0-b4d8-4187-9d0a-b525bd7a1978 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:05.396484 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 61/122] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:13:05 2022] GET /identity/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:05.397746 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec9a65-cd78-466e-84dc-5d63d24918b8 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:05.402686 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec9a65-cd78-466e-84dc-5d63d24918b8 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:05.403114 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec9a65-cd78-466e-84dc-5d63d24918b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:05.424079 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-9aec9a65-cd78-466e-84dc-5d63d24918b8 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:05.425424 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9aec9a65-cd78-466e-84dc-5d63d24918b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:05.502443 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 62/123] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:05.504028 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c0472-0008-45ba-9719-d2a59300ae8e None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:05.504384 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c0472-0008-45ba-9719-d2a59300ae8e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:05.504729 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c0472-0008-45ba-9719-d2a59300ae8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:05.527849 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-de5c0472-0008-45ba-9719-d2a59300ae8e None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:05.531241 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922757d-fb0a-41b1-9e32-c87ce28314d6 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:05.532921 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-de5c0472-0008-45ba-9719-d2a59300ae8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:05.569728 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e2ef3-7861-4d7f-b9f0-319f319e5df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:05.570235 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e2ef3-7861-4d7f-b9f0-319f319e5df7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:05.570520 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e2ef3-7861-4d7f-b9f0-319f319e5df7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:05.570836 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e2ef3-7861-4d7f-b9f0-319f319e5df7 None admin] PATH_INFO: `/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:05.572066 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2e2ef3-7861-4d7f-b9f0-319f319e5df7 None admin] RBAC: Authorizing `identity:get_project(project_id=a9d326f0c0fa4a38aa7ba8c8c6797c5a)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:05.573913 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2e2ef3-7861-4d7f-b9f0-319f319e5df7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:05.575868 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 62/124] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:13:05 2022] GET /identity/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:05.581526 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e2ef3-7861-4d7f-b9f0-319f319e5df7 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:05.619850 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:05.621408 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 63/125] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:05.629161 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5c0472-0008-45ba-9719-d2a59300ae8e None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:05.669294 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:05.715424 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e2ef3-7861-4d7f-b9f0-319f319e5df7 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:05.756891 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5c0472-0008-45ba-9719-d2a59300ae8e None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:05.761951 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd02b3d0-076a-4189-9cdc-ebb6ba6a8789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:05.762707 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02b3d0-076a-4189-9cdc-ebb6ba6a8789 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:05.763035 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02b3d0-076a-4189-9cdc-ebb6ba6a8789 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:05.763320 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02b3d0-076a-4189-9cdc-ebb6ba6a8789 None admin] PATH_INFO: `/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a/users/898d0645ae1e477983200e39184fa9ed/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:05.765365 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd02b3d0-076a-4189-9cdc-ebb6ba6a8789 None admin] RBAC: Authorizing `identity:create_grant(project_id=a9d326f0c0fa4a38aa7ba8c8c6797c5a, user_id=898d0645ae1e477983200e39184fa9ed, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:05.766997 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd02b3d0-076a-4189-9cdc-ebb6ba6a8789 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:05.786853 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 63/126] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 10:13:05 2022] PUT /identity/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a/users/898d0645ae1e477983200e39184fa9ed/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:13:05.791577 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd02b3d0-076a-4189-9cdc-ebb6ba6a8789 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:05.818381 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5272d868-3ff7-4eae-87ba-db8444c1ce2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:05.819500 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5272d868-3ff7-4eae-87ba-db8444c1ce2c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:05.820004 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5272d868-3ff7-4eae-87ba-db8444c1ce2c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:05.820470 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5272d868-3ff7-4eae-87ba-db8444c1ce2c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:05.838374 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5272d868-3ff7-4eae-87ba-db8444c1ce2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:05.844800 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-5272d868-3ff7-4eae-87ba-db8444c1ce2c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 10:13:05.844800 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:05.844800 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:05.844800 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:05.844800 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:05.844800 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:05.844800 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:05.844800 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:05.844800 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:05.844800 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:05.844800 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:05.844800 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:05.844800 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:05.844800 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:05.844800 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:05.844800 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:05.845694 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:05.845694 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:05.845694 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:05.845694 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:05.845694 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:05.845694 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:05.845694 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:05.845694 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:05.845694 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:05.845694 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:05.845694 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:05.845694 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:05.845694 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:05.845694 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:05.845694 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:05.845694 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:05.846431 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:05.846431 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:05.846431 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:05.846431 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:05.846431 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:05.846431 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:05.846431 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:05.846431 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:05.846431 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:05.846431 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:05.846431 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 10:13:05.846431 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:13:05.849095 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 64/127] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:13:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:05.850273 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5272d868-3ff7-4eae-87ba-db8444c1ce2c None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:06.012701 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:06.020478 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:06.030794 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd02b3d0-076a-4189-9cdc-ebb6ba6a8789 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:06.038566 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5272d868-3ff7-4eae-87ba-db8444c1ce2c None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:06.082202 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94239fba-3888-427d-b1db-2f389e443e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:06.083043 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94239fba-3888-427d-b1db-2f389e443e0c None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:06.083336 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94239fba-3888-427d-b1db-2f389e443e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:06.083618 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94239fba-3888-427d-b1db-2f389e443e0c None admin] PATH_INFO: `/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a/users/f720a2ce258149afacb65b2c45fa2173/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:06.085812 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e0c183-1deb-403a-af39-c8c207bfc3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:06.086004 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94239fba-3888-427d-b1db-2f389e443e0c None admin] RBAC: Authorizing `identity:create_grant(project_id=a9d326f0c0fa4a38aa7ba8c8c6797c5a, user_id=f720a2ce258149afacb65b2c45fa2173, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:06.086841 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e0c183-1deb-403a-af39-c8c207bfc3c6 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:06.087337 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e0c183-1deb-403a-af39-c8c207bfc3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:06.087522 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94239fba-3888-427d-b1db-2f389e443e0c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:06.087988 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e0c183-1deb-403a-af39-c8c207bfc3c6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:06.103937 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e0c183-1deb-403a-af39-c8c207bfc3c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:06.108682 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 64/128] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 10:13:05 2022] PUT /identity/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a/users/f720a2ce258149afacb65b2c45fa2173/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:13:06.110062 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94239fba-3888-427d-b1db-2f389e443e0c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:06.111085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-43e0c183-1deb-403a-af39-c8c207bfc3c6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 10:13:06.111085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:06.111085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:06.111085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:06.111085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:06.111085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:06.111085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:06.111085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:06.111085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:06.111085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:06.111085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:06.111085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:06.111085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:06.111085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:06.111085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:06.111085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:06.112190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:06.112190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:06.112190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:06.112190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:06.112190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:06.112190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:06.112190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:06.112190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:06.112190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:06.112190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:06.112190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:06.112190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:06.112190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:06.112190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:06.112190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:06.112190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:06.112966 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:06.112966 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:06.112966 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:06.112966 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:06.112966 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:06.112966 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:06.112966 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:06.112966 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:06.112966 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:06.112966 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:06.112966 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 10:13:06.112966 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:13:06.115786 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 65/129] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 5 10:13:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:06.121858 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e0c183-1deb-403a-af39-c8c207bfc3c6 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:06.252352 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e0c183-1deb-403a-af39-c8c207bfc3c6 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:06.263865 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94239fba-3888-427d-b1db-2f389e443e0c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:06.290431 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcdd277-9833-4606-813e-b6ba01abd2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:06.291496 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcdd277-9833-4606-813e-b6ba01abd2cd None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:06.291969 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcdd277-9833-4606-813e-b6ba01abd2cd None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:06.292428 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcdd277-9833-4606-813e-b6ba01abd2cd None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:06.293331 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcdd277-9833-4606-813e-b6ba01abd2cd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:06.293819 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcdd277-9833-4606-813e-b6ba01abd2cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:06.295787 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcdd277-9833-4606-813e-b6ba01abd2cd None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:06.305503 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab35744b-de55-4286-8063-bfbde6e3eab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:06.306228 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35744b-de55-4286-8063-bfbde6e3eab8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:06.306518 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35744b-de55-4286-8063-bfbde6e3eab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:06.306798 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35744b-de55-4286-8063-bfbde6e3eab8 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:06.307647 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab35744b-de55-4286-8063-bfbde6e3eab8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:06.308133 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab35744b-de55-4286-8063-bfbde6e3eab8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:06.308559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 66/130] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:13:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:06.313157 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab35744b-de55-4286-8063-bfbde6e3eab8 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:06.314362 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcdd277-9833-4606-813e-b6ba01abd2cd None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:06.323363 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 65/131] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 10:13:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:06.328501 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:06.330089 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab35744b-de55-4286-8063-bfbde6e3eab8 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:06.351820 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:06.428424 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcdd277-9833-4606-813e-b6ba01abd2cd None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:06.458426 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab35744b-de55-4286-8063-bfbde6e3eab8 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:06.465536 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046618d3-c892-4b71-9066-7a1b99e6c17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:06.466539 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046618d3-c892-4b71-9066-7a1b99e6c17c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:06.467061 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046618d3-c892-4b71-9066-7a1b99e6c17c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:06.467666 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046618d3-c892-4b71-9066-7a1b99e6c17c None admin] PATH_INFO: `/v3/users/f720a2ce258149afacb65b2c45fa2173` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:06.469142 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046618d3-c892-4b71-9066-7a1b99e6c17c None admin] RBAC: Authorizing `identity:get_user(user_id=f720a2ce258149afacb65b2c45fa2173)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:06.470871 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046618d3-c892-4b71-9066-7a1b99e6c17c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:06.472812 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 67/132] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:13:06 2022] GET /identity/v3/users/f720a2ce258149afacb65b2c45fa2173 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:06.474234 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5607fb27-cd35-40d3-b4ec-b15c6305e7d0 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:06.474714 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5607fb27-cd35-40d3-b4ec-b15c6305e7d0 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:06.475198 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5607fb27-cd35-40d3-b4ec-b15c6305e7d0 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:06.476877 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 68/133] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:06.482464 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73ab9eb-ca1f-4797-89b4-a076078c8a0c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:06.483043 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73ab9eb-ca1f-4797-89b4-a076078c8a0c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:06.483505 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73ab9eb-ca1f-4797-89b4-a076078c8a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:06.503047 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-f73ab9eb-ca1f-4797-89b4-a076078c8a0c None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:06.504566 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f73ab9eb-ca1f-4797-89b4-a076078c8a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:06.505053 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8213dfa-ed65-4fc8-a3ec-fa74e3ae0f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:06.506049 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8213dfa-ed65-4fc8-a3ec-fa74e3ae0f9c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:06.506573 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8213dfa-ed65-4fc8-a3ec-fa74e3ae0f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:06.507089 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8213dfa-ed65-4fc8-a3ec-fa74e3ae0f9c None admin] PATH_INFO: `/v3/users/f720a2ce258149afacb65b2c45fa2173` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:06.510162 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8213dfa-ed65-4fc8-a3ec-fa74e3ae0f9c None admin] RBAC: Authorizing `identity:get_user(user_id=f720a2ce258149afacb65b2c45fa2173)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:06.512049 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8213dfa-ed65-4fc8-a3ec-fa74e3ae0f9c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:06.514687 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 66/134] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:13:06 2022] GET /identity/v3/users/f720a2ce258149afacb65b2c45fa2173 => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:13:06.516020 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8213dfa-ed65-4fc8-a3ec-fa74e3ae0f9c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:06.584130 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 69/135] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:06.585008 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73ab9eb-ca1f-4797-89b4-a076078c8a0c None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:06.592804 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:06.650855 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:06.677707 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8213dfa-ed65-4fc8-a3ec-fa74e3ae0f9c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:06.703054 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73ab9eb-ca1f-4797-89b4-a076078c8a0c None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:06.726606 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e258d06f-2a90-42a0-b48b-120db6f063d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:06.728045 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258d06f-2a90-42a0-b48b-120db6f063d0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:06.729058 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258d06f-2a90-42a0-b48b-120db6f063d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:06.730066 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258d06f-2a90-42a0-b48b-120db6f063d0 None admin] PATH_INFO: `/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:06.732210 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e258d06f-2a90-42a0-b48b-120db6f063d0 None admin] RBAC: Authorizing `identity:get_project(project_id=a9d326f0c0fa4a38aa7ba8c8c6797c5a)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:06.734990 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e258d06f-2a90-42a0-b48b-120db6f063d0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:06.737843 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 67/136] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:13:06 2022] GET /identity/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:06.739070 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e258d06f-2a90-42a0-b48b-120db6f063d0 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:06.742039 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ed471-13b9-4e04-8d7e-2793458ca05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:06.742707 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ed471-13b9-4e04-8d7e-2793458ca05e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:06.743019 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ed471-13b9-4e04-8d7e-2793458ca05e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:06.743293 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ed471-13b9-4e04-8d7e-2793458ca05e None admin] PATH_INFO: `/v3/projects/387614ebbd51447b89e7a0c2ee6d4b96` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:06.744482 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672ed471-13b9-4e04-8d7e-2793458ca05e None admin] RBAC: Authorizing `identity:get_project(project_id=387614ebbd51447b89e7a0c2ee6d4b96)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:06.745942 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672ed471-13b9-4e04-8d7e-2793458ca05e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:06.747697 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 70/137] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:13:06 2022] GET /identity/v3/projects/387614ebbd51447b89e7a0c2ee6d4b96 => generated 312 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:06.748434 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ed471-13b9-4e04-8d7e-2793458ca05e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:06.831655 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:06.879846 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ed471-13b9-4e04-8d7e-2793458ca05e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:06.926006 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7538ae9b-1012-427e-ac55-31f1d9b16805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:06.926713 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7538ae9b-1012-427e-ac55-31f1d9b16805 None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:06.927098 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7538ae9b-1012-427e-ac55-31f1d9b16805 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:06.927673 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7538ae9b-1012-427e-ac55-31f1d9b16805 None admin] PATH_INFO: `/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a/users/f720a2ce258149afacb65b2c45fa2173/roles/57a070b43ad940f1886c69721dd304c2` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:06.929634 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7538ae9b-1012-427e-ac55-31f1d9b16805 None admin] RBAC: Authorizing `identity:create_grant(project_id=a9d326f0c0fa4a38aa7ba8c8c6797c5a, user_id=f720a2ce258149afacb65b2c45fa2173, role_id=57a070b43ad940f1886c69721dd304c2)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:06.931316 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7538ae9b-1012-427e-ac55-31f1d9b16805 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:06.933936 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e258d06f-2a90-42a0-b48b-120db6f063d0 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:06.948352 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 71/138] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 10:13:06 2022] PUT /identity/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a/users/f720a2ce258149afacb65b2c45fa2173/roles/57a070b43ad940f1886c69721dd304c2 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:13:06.949224 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7538ae9b-1012-427e-ac55-31f1d9b16805 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:07.060376 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922285a8-ca1f-44c5-803e-22d57ca7bf8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:07.061128 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922285a8-ca1f-44c5-803e-22d57ca7bf8e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:07.061414 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922285a8-ca1f-44c5-803e-22d57ca7bf8e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:07.061685 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922285a8-ca1f-44c5-803e-22d57ca7bf8e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:07.065269 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922285a8-ca1f-44c5-803e-22d57ca7bf8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:07.066772 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922285a8-ca1f-44c5-803e-22d57ca7bf8e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:07.068507 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 68/139] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:13:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:07.079982 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922285a8-ca1f-44c5-803e-22d57ca7bf8e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:07.091039 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7538ae9b-1012-427e-ac55-31f1d9b16805 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:07.131497 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:07.137085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53846c21-fee1-4b35-b557-e5bdb939d4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:07.137858 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53846c21-fee1-4b35-b557-e5bdb939d4dc None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:07.138284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53846c21-fee1-4b35-b557-e5bdb939d4dc None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:07.138800 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53846c21-fee1-4b35-b557-e5bdb939d4dc None admin] PATH_INFO: `/v3/projects/387614ebbd51447b89e7a0c2ee6d4b96/users/f720a2ce258149afacb65b2c45fa2173/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:07.143643 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53846c21-fee1-4b35-b557-e5bdb939d4dc None admin] RBAC: Authorizing `identity:create_grant(project_id=387614ebbd51447b89e7a0c2ee6d4b96, user_id=f720a2ce258149afacb65b2c45fa2173, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:07.145908 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53846c21-fee1-4b35-b557-e5bdb939d4dc None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:07.163001 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:07.194805 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 72/140] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 10:13:06 2022] PUT /identity/v3/projects/387614ebbd51447b89e7a0c2ee6d4b96/users/f720a2ce258149afacb65b2c45fa2173/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:13:07.302159 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922285a8-ca1f-44c5-803e-22d57ca7bf8e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:07.352104 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eec6de-78dd-4677-8763-148a551fa613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:07.352850 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eec6de-78dd-4677-8763-148a551fa613 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:07.353136 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eec6de-78dd-4677-8763-148a551fa613 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:07.353425 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eec6de-78dd-4677-8763-148a551fa613 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:07.354239 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eec6de-78dd-4677-8763-148a551fa613 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:07.355834 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eec6de-78dd-4677-8763-148a551fa613 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:07.395254 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae56f11c-110d-4b78-93bb-b812b7e7b8f1 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:07.395571 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae56f11c-110d-4b78-93bb-b812b7e7b8f1 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:07.396079 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae56f11c-110d-4b78-93bb-b812b7e7b8f1 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:07.397712 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 73/141] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:07.403631 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc8d57-d50b-4bd1-8c98-7a7e338a49c4 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:07.403919 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc8d57-d50b-4bd1-8c98-7a7e338a49c4 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:07.404235 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc8d57-d50b-4bd1-8c98-7a7e338a49c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:07.414967 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 69/142] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 5 10:13:07 2022] POST /identity/v3/users => generated 283 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:13:07.433362 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-cbdc8d57-d50b-4bd1-8c98-7a7e338a49c4 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:07.434650 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-cbdc8d57-d50b-4bd1-8c98-7a7e338a49c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:07.529198 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 74/143] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 10:13:07.577876 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eec6de-78dd-4677-8763-148a551fa613 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:07.670631 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:07.697854 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eec6de-78dd-4677-8763-148a551fa613 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:07.734744 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a2d04-acff-4861-a097-4a553ab74495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:07.735488 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a2d04-acff-4861-a097-4a553ab74495 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:07.735772 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a2d04-acff-4861-a097-4a553ab74495 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:07.736089 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a2d04-acff-4861-a097-4a553ab74495 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:07.746831 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12a2d04-acff-4861-a097-4a553ab74495 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:07.752942 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-d12a2d04-acff-4861-a097-4a553ab74495 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 10:13:07.752942 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:07.752942 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:07.752942 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:07.752942 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:07.752942 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:07.752942 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:07.752942 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:07.752942 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:07.752942 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:07.752942 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:07.752942 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:07.752942 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:07.752942 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:07.752942 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:07.752942 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:07.753836 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:07.753836 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:07.753836 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:07.753836 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:07.753836 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:07.753836 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:07.753836 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:07.753836 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:07.753836 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:07.753836 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:07.753836 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:07.753836 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:07.753836 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:07.753836 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:07.753836 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:07.753836 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:07.754592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:07.754592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:07.754592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:07.754592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:07.754592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:07.754592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:07.754592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:07.754592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:07.754592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:07.754592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:07.754592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 10:13:07.754592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:07.754592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 70/144] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:13:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:07.760172 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdc8d57-d50b-4bd1-8c98-7a7e338a49c4 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:07.868395 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdc8d57-d50b-4bd1-8c98-7a7e338a49c4 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:07.896896 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:07.908931 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e98df-14a4-4a60-9dc0-089be4764e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:07.909715 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e98df-14a4-4a60-9dc0-089be4764e35 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:07.910000 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e98df-14a4-4a60-9dc0-089be4764e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:07.910278 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e98df-14a4-4a60-9dc0-089be4764e35 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:07.911169 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52e98df-14a4-4a60-9dc0-089be4764e35 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:07.915335 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52e98df-14a4-4a60-9dc0-089be4764e35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:07.917235 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52e98df-14a4-4a60-9dc0-089be4764e35 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:07.929658 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 75/145] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 10:13:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:07.935464 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a2d04-acff-4861-a097-4a553ab74495 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:08.047269 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a2d04-acff-4861-a097-4a553ab74495 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:08.085319 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee707dd-4aa2-4236-a931-23084751dc91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:08.086087 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee707dd-4aa2-4236-a931-23084751dc91 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:08.086246 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee707dd-4aa2-4236-a931-23084751dc91 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:08.086522 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee707dd-4aa2-4236-a931-23084751dc91 None admin] PATH_INFO: `/v3/users/dd861837dc1d4bb2a5bcc19c545b8432` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:08.087725 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee707dd-4aa2-4236-a931-23084751dc91 None admin] RBAC: Authorizing `identity:get_user(user_id=dd861837dc1d4bb2a5bcc19c545b8432)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:08.089307 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee707dd-4aa2-4236-a931-23084751dc91 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:08.091431 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 71/146] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:13:07 2022] GET /identity/v3/users/dd861837dc1d4bb2a5bcc19c545b8432 => generated 289 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:08.098459 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e98df-14a4-4a60-9dc0-089be4764e35 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:08.235818 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e98df-14a4-4a60-9dc0-089be4764e35 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:08.274500 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af1de9-d533-4a85-adf8-67e7121d3abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:08.275191 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af1de9-d533-4a85-adf8-67e7121d3abc None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:08.275463 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af1de9-d533-4a85-adf8-67e7121d3abc None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:08.275732 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af1de9-d533-4a85-adf8-67e7121d3abc None admin] PATH_INFO: `/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:08.277035 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53af1de9-d533-4a85-adf8-67e7121d3abc None admin] RBAC: Authorizing `identity:get_project(project_id=a9d326f0c0fa4a38aa7ba8c8c6797c5a)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:08.278484 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53af1de9-d533-4a85-adf8-67e7121d3abc None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:08.280192 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 76/147] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:13:08 2022] GET /identity/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:08.286339 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee707dd-4aa2-4236-a931-23084751dc91 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:08.297366 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:08.418436 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee707dd-4aa2-4236-a931-23084751dc91 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:08.455510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437fe234-b799-41a5-8516-8a7f68728823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:08.456205 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437fe234-b799-41a5-8516-8a7f68728823 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:08.456487 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437fe234-b799-41a5-8516-8a7f68728823 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:08.456766 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437fe234-b799-41a5-8516-8a7f68728823 None admin] PATH_INFO: `/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a/users/dd861837dc1d4bb2a5bcc19c545b8432/roles/cf5de414a035475b8b75626cb19a2de2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:08.463748 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437fe234-b799-41a5-8516-8a7f68728823 None admin] RBAC: Authorizing `identity:create_grant(project_id=a9d326f0c0fa4a38aa7ba8c8c6797c5a, user_id=dd861837dc1d4bb2a5bcc19c545b8432, role_id=cf5de414a035475b8b75626cb19a2de2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:08.465243 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437fe234-b799-41a5-8516-8a7f68728823 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:08.492624 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 72/148] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 10:13:08 2022] PUT /identity/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a/users/dd861837dc1d4bb2a5bcc19c545b8432/roles/cf5de414a035475b8b75626cb19a2de2 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:13:08.888943 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140c190-8054-482a-9b9a-be3b5fc40d2c None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:08.889277 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140c190-8054-482a-9b9a-be3b5fc40d2c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:08.889399 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140c190-8054-482a-9b9a-be3b5fc40d2c None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:08.890880 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 77/149] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:08.897305 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01505d8-f7e4-4d78-9da3-23808b83621c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:08.897842 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01505d8-f7e4-4d78-9da3-23808b83621c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:08.898305 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01505d8-f7e4-4d78-9da3-23808b83621c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:08.919573 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b01505d8-f7e4-4d78-9da3-23808b83621c None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:08.921504 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b01505d8-f7e4-4d78-9da3-23808b83621c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:08.995893 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:09.011288 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 73/150] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:09.054032 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140c190-8054-482a-9b9a-be3b5fc40d2c None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:09.118072 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:09.137573 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab8ace-8935-4f49-92b2-8f76e33fd63b None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:09.138078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab8ace-8935-4f49-92b2-8f76e33fd63b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:09.138149 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab8ace-8935-4f49-92b2-8f76e33fd63b None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:09.139681 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 74/151] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 10:13:09.146010 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b15aa6-5d97-4a7f-8ef9-76dbc7add082 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:09.146289 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b15aa6-5d97-4a7f-8ef9-76dbc7add082 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:09.146554 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b15aa6-5d97-4a7f-8ef9-76dbc7add082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:09.178628 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a4b15aa6-5d97-4a7f-8ef9-76dbc7add082 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:09.180541 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a4b15aa6-5d97-4a7f-8ef9-76dbc7add082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:09.220716 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140c190-8054-482a-9b9a-be3b5fc40d2c None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:09.256339 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 75/152] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 10:13:09.280174 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6ff499-c240-4162-9d22-afddf4463223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:09.280925 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ff499-c240-4162-9d22-afddf4463223 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:09.281213 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ff499-c240-4162-9d22-afddf4463223 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:09.281493 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ff499-c240-4162-9d22-afddf4463223 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:09.297906 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b15aa6-5d97-4a7f-8ef9-76dbc7add082 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:09.302117 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6ff499-c240-4162-9d22-afddf4463223 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:09.309327 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-0d6ff499-c240-4162-9d22-afddf4463223 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:13:09.309327 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:09.309327 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:09.309327 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:09.309327 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:09.309327 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:09.309327 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:09.309327 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:09.309327 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:09.309327 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:09.309327 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:09.309327 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:09.309327 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:09.309327 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:09.309327 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:09.309327 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:09.310400 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:09.310400 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:09.310400 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:09.310400 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:09.310400 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:09.310400 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:09.310400 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:09.310400 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:09.310400 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:09.310400 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:09.310400 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:09.310400 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:09.310400 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:09.310400 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:09.310400 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:09.310400 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:09.311398 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:09.311398 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:09.311398 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:09.311398 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:09.311398 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:09.311398 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:09.311398 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:09.311398 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:09.311398 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:09.311398 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:09.311398 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:13:09.311398 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:13:09.311398 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 78/153] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 10:13:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:09.320028 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6ff499-c240-4162-9d22-afddf4463223 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:09.499350 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6ff499-c240-4162-9d22-afddf4463223 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:09.506743 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b15aa6-5d97-4a7f-8ef9-76dbc7add082 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:09.547432 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:09.548514 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:09.555965 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425ff1e-e457-4247-ae1e-3e0e5c2509e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:09.556613 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425ff1e-e457-4247-ae1e-3e0e5c2509e6 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:09.556899 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425ff1e-e457-4247-ae1e-3e0e5c2509e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:09.557176 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425ff1e-e457-4247-ae1e-3e0e5c2509e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:09.557906 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a425ff1e-e457-4247-ae1e-3e0e5c2509e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:09.558209 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a425ff1e-e457-4247-ae1e-3e0e5c2509e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:09.559685 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a425ff1e-e457-4247-ae1e-3e0e5c2509e6 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:09.566398 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458c816d-6116-4af0-b609-619413b5ef13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:09.567110 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c816d-6116-4af0-b609-619413b5ef13 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:09.567397 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c816d-6116-4af0-b609-619413b5ef13 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:09.567669 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c816d-6116-4af0-b609-619413b5ef13 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:09.573664 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 79/154] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:13:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:09.582695 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425ff1e-e457-4247-ae1e-3e0e5c2509e6 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:09.584931 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458c816d-6116-4af0-b609-619413b5ef13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:09.594203 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-458c816d-6116-4af0-b609-619413b5ef13 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 10:13:09.594203 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:09.594203 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:09.594203 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:09.594203 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:09.594203 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:09.594203 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:09.594203 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:09.594203 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:09.594203 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:09.594203 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:09.594203 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:09.594203 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:09.594203 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:09.594203 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:09.594203 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:09.595127 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:09.595127 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:09.595127 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:09.595127 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:09.595127 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:09.595127 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:09.595127 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:09.595127 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:09.595127 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:09.595127 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:09.595127 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:09.595127 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:09.595127 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:09.595127 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:09.595127 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:09.595127 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:09.596746 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:09.596746 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:09.596746 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:09.596746 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:09.596746 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:09.596746 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:09.596746 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:09.596746 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:09.596746 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:09.596746 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:09.596746 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 10:13:09.596746 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:09.596746 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 76/155] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:13:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:13:09.604884 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458c816d-6116-4af0-b609-619413b5ef13 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:09.796072 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458c816d-6116-4af0-b609-619413b5ef13 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:09.831217 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425ff1e-e457-4247-ae1e-3e0e5c2509e6 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:09.877502 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce1bdca-e2c0-468f-b82a-0c2081bd12e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:09.878142 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1bdca-e2c0-468f-b82a-0c2081bd12e8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:09.878428 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1bdca-e2c0-468f-b82a-0c2081bd12e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:09.878687 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1bdca-e2c0-468f-b82a-0c2081bd12e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:09.879597 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce1bdca-e2c0-468f-b82a-0c2081bd12e8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:09.879902 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce1bdca-e2c0-468f-b82a-0c2081bd12e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:09.881310 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce1bdca-e2c0-468f-b82a-0c2081bd12e8 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:09.901336 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 77/156] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 10:13:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:09.907908 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce1bdca-e2c0-468f-b82a-0c2081bd12e8 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:09.908703 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dea9bd-05f7-4364-91d9-655952f7fd21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:09.909363 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dea9bd-05f7-4364-91d9-655952f7fd21 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:09.909642 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dea9bd-05f7-4364-91d9-655952f7fd21 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:09.909909 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dea9bd-05f7-4364-91d9-655952f7fd21 None admin] PATH_INFO: `/v3/users/898d0645ae1e477983200e39184fa9ed` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:09.911417 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dea9bd-05f7-4364-91d9-655952f7fd21 None admin] RBAC: Authorizing `identity:get_user(user_id=898d0645ae1e477983200e39184fa9ed)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:09.912875 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dea9bd-05f7-4364-91d9-655952f7fd21 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:09.914485 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 80/157] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:13:09 2022] GET /identity/v3/users/898d0645ae1e477983200e39184fa9ed => generated 247 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:09.931518 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dea9bd-05f7-4364-91d9-655952f7fd21 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:09.997055 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:10.005680 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:10.092601 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce1bdca-e2c0-468f-b82a-0c2081bd12e8 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:10.105775 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dea9bd-05f7-4364-91d9-655952f7fd21 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:10.152858 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86f165-9721-4760-a5b8-89be71ddeefc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:10.153522 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86f165-9721-4760-a5b8-89be71ddeefc None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:10.153794 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86f165-9721-4760-a5b8-89be71ddeefc None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:10.154070 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86f165-9721-4760-a5b8-89be71ddeefc None admin] PATH_INFO: `/v3/users/f720a2ce258149afacb65b2c45fa2173` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:10.155726 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a86f165-9721-4760-a5b8-89be71ddeefc None admin] RBAC: Authorizing `identity:get_user(user_id=f720a2ce258149afacb65b2c45fa2173)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:10.157459 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a86f165-9721-4760-a5b8-89be71ddeefc None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:10.160383 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 78/158] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:13:09 2022] GET /identity/v3/users/f720a2ce258149afacb65b2c45fa2173 => generated 275 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:10.166938 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5678131c-68e6-46d2-8e43-7c027ced4694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:10.167419 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86f165-9721-4760-a5b8-89be71ddeefc None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:10.167503 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678131c-68e6-46d2-8e43-7c027ced4694 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:10.167503 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678131c-68e6-46d2-8e43-7c027ced4694 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:10.167503 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678131c-68e6-46d2-8e43-7c027ced4694 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:10.285584 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5678131c-68e6-46d2-8e43-7c027ced4694 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:10.287170 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5678131c-68e6-46d2-8e43-7c027ced4694 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:10.341115 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86f165-9721-4760-a5b8-89be71ddeefc None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:10.360405 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:10.367662 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:10.405315 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f65807f-5502-40c0-b3a4-fcab20016801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:10.406116 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f65807f-5502-40c0-b3a4-fcab20016801 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:10.406455 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f65807f-5502-40c0-b3a4-fcab20016801 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:10.406733 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f65807f-5502-40c0-b3a4-fcab20016801 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:10.481508 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 81/159] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:13:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 10:13:10.482481 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f65807f-5502-40c0-b3a4-fcab20016801 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:10.482792 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a6084-3c9f-4649-8894-7d14f4fb2105 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:10.483119 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a6084-3c9f-4649-8894-7d14f4fb2105 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:10.483391 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a6084-3c9f-4649-8894-7d14f4fb2105 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:10.484081 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f65807f-5502-40c0-b3a4-fcab20016801 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:10.485032 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 82/160] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:10.490871 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b452c07-e268-419b-abb6-79ef4d5850f1 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:10.491198 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b452c07-e268-419b-abb6-79ef4d5850f1 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:10.491475 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b452c07-e268-419b-abb6-79ef4d5850f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:10.540018 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-2b452c07-e268-419b-abb6-79ef4d5850f1 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:10.541215 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2b452c07-e268-419b-abb6-79ef4d5850f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:10.645689 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 79/161] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:13:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 10:13:10.646476 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f65807f-5502-40c0-b3a4-fcab20016801 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:10.672257 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 83/162] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:10.674796 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b452c07-e268-419b-abb6-79ef4d5850f1 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:10.679267 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:10.831404 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f65807f-5502-40c0-b3a4-fcab20016801 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:10.843148 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:10.899987 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3da57-a87b-4e13-b479-6741c94d6604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:10.900705 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3da57-a87b-4e13-b479-6741c94d6604 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:10.901007 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3da57-a87b-4e13-b479-6741c94d6604 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:10.901287 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3da57-a87b-4e13-b479-6741c94d6604 None admin] PATH_INFO: `/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:10.902748 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c3da57-a87b-4e13-b479-6741c94d6604 None admin] RBAC: Authorizing `identity:get_project(project_id=a9d326f0c0fa4a38aa7ba8c8c6797c5a)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:10.904520 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c3da57-a87b-4e13-b479-6741c94d6604 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:10.905723 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b452c07-e268-419b-abb6-79ef4d5850f1 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:10.906256 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 80/163] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:13:10 2022] GET /identity/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a => generated 298 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:10.907222 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3da57-a87b-4e13-b479-6741c94d6604 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:10.984534 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd23142-6bb1-45ac-9813-77a172d126f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:10.985198 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd23142-6bb1-45ac-9813-77a172d126f1 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:10.985476 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd23142-6bb1-45ac-9813-77a172d126f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:10.985746 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd23142-6bb1-45ac-9813-77a172d126f1 None admin] PATH_INFO: `/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:10.991805 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd23142-6bb1-45ac-9813-77a172d126f1 None admin] RBAC: Authorizing `identity:get_project(project_id=a9d326f0c0fa4a38aa7ba8c8c6797c5a)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:10.993446 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd23142-6bb1-45ac-9813-77a172d126f1 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:10.995343 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 84/164] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:13:10 2022] GET /identity/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a => generated 298 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:10.996231 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd23142-6bb1-45ac-9813-77a172d126f1 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:11.080317 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:11.119775 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3da57-a87b-4e13-b479-6741c94d6604 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:11.217631 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfb6594-0d51-4b0b-82ed-c024e1dd8184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:11.218322 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb6594-0d51-4b0b-82ed-c024e1dd8184 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:11.218610 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb6594-0d51-4b0b-82ed-c024e1dd8184 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:11.218880 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb6594-0d51-4b0b-82ed-c024e1dd8184 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:11.244609 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfb6594-0d51-4b0b-82ed-c024e1dd8184 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:11.249999 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-5bfb6594-0d51-4b0b-82ed-c024e1dd8184 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 10:13:11.249999 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:11.249999 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:11.249999 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:11.249999 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:11.249999 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:11.249999 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:11.249999 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:11.249999 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:11.249999 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:11.249999 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:11.249999 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:11.249999 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:11.249999 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:11.249999 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:11.249999 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:11.250889 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:11.250889 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:11.250889 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:11.250889 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:11.250889 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:11.250889 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:11.250889 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:11.250889 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:11.250889 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:11.250889 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:11.250889 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:11.250889 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:11.250889 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:11.250889 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:11.250889 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:11.250889 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:11.252129 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:11.252129 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:11.252129 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:11.252129 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:11.252129 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:11.252129 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:11.252129 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:11.252129 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:11.252129 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:11.252129 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:11.252129 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 10:13:11.252129 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:11.252129 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 81/165] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 5 10:13:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:11.253060 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfb6594-0d51-4b0b-82ed-c024e1dd8184 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:11.268748 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd23142-6bb1-45ac-9813-77a172d126f1 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:11.322714 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfeb39e-73c1-4153-8013-dd41f64bd644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:11.323376 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfeb39e-73c1-4153-8013-dd41f64bd644 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:11.323655 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfeb39e-73c1-4153-8013-dd41f64bd644 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:11.323914 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfeb39e-73c1-4153-8013-dd41f64bd644 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:11.324712 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfeb39e-73c1-4153-8013-dd41f64bd644 None admin] RBAC: Adding query filter params (role.id=daef89adcaeb48a382a0bcb03340ccbb, scope.project.id=a9d326f0c0fa4a38aa7ba8c8c6797c5a, user.id=898d0645ae1e477983200e39184fa9ed) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:11.325017 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfeb39e-73c1-4153-8013-dd41f64bd644 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:11.326489 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfeb39e-73c1-4153-8013-dd41f64bd644 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:11.333102 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 85/166] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:13:10 2022] GET /identity/v3/role_assignments?user.id=898d0645ae1e477983200e39184fa9ed&scope.project.id=a9d326f0c0fa4a38aa7ba8c8c6797c5a&role.id=daef89adcaeb48a382a0bcb03340ccbb => generated 608 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:11.333915 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfeb39e-73c1-4153-8013-dd41f64bd644 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:11.342945 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:11.460062 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfb6594-0d51-4b0b-82ed-c024e1dd8184 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:11.479173 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfeb39e-73c1-4153-8013-dd41f64bd644 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:11.522357 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7025d64e-d295-441f-b5e5-742136702c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:11.523022 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025d64e-d295-441f-b5e5-742136702c50 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:11.523302 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025d64e-d295-441f-b5e5-742136702c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:11.523570 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025d64e-d295-441f-b5e5-742136702c50 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:11.524996 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7025d64e-d295-441f-b5e5-742136702c50 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:11.525312 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7025d64e-d295-441f-b5e5-742136702c50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:11.526764 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7025d64e-d295-441f-b5e5-742136702c50 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:11.540647 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 86/167] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:13:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:11.541944 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afada9f-73ee-45f9-a4f5-602e1eedb145 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:11.542225 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afada9f-73ee-45f9-a4f5-602e1eedb145 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:11.542494 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afada9f-73ee-45f9-a4f5-602e1eedb145 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:11.544053 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 87/168] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:11.544753 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68480c9-06f2-4b82-9cfc-276cc590e201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:11.545788 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68480c9-06f2-4b82-9cfc-276cc590e201 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:11.546073 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68480c9-06f2-4b82-9cfc-276cc590e201 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:11.546379 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68480c9-06f2-4b82-9cfc-276cc590e201 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:11.547335 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68480c9-06f2-4b82-9cfc-276cc590e201 None admin] RBAC: Adding query filter params (role.id=c4ba94fa4bd34a09ba87d357142d9db8, scope.project.id=a9d326f0c0fa4a38aa7ba8c8c6797c5a, user.id=f720a2ce258149afacb65b2c45fa2173) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:11.547579 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68480c9-06f2-4b82-9cfc-276cc590e201 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:11.549535 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68480c9-06f2-4b82-9cfc-276cc590e201 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:11.550199 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103434e-35db-4d99-9129-a157062dcc5d None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:11.550485 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103434e-35db-4d99-9129-a157062dcc5d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:11.550758 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103434e-35db-4d99-9129-a157062dcc5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:11.565470 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 82/169] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:13:11 2022] GET /identity/v3/role_assignments?user.id=f720a2ce258149afacb65b2c45fa2173&scope.project.id=a9d326f0c0fa4a38aa7ba8c8c6797c5a&role.id=c4ba94fa4bd34a09ba87d357142d9db8 => generated 608 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:11.566139 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68480c9-06f2-4b82-9cfc-276cc590e201 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:11.581432 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:11.583154 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-6103434e-35db-4d99-9129-a157062dcc5d None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:11.584452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6103434e-35db-4d99-9129-a157062dcc5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:11.684943 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:11.701928 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 88/170] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 10:13:11.737951 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103434e-35db-4d99-9129-a157062dcc5d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:11.841045 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68480c9-06f2-4b82-9cfc-276cc590e201 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:11.903782 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103434e-35db-4d99-9129-a157062dcc5d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:11.914642 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f53175-0dec-4170-bb52-7ae5aa9f703d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:11.915704 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f53175-0dec-4170-bb52-7ae5aa9f703d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:11.916213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f53175-0dec-4170-bb52-7ae5aa9f703d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:11.916671 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f53175-0dec-4170-bb52-7ae5aa9f703d None admin] PATH_INFO: `/v3/users/f720a2ce258149afacb65b2c45fa2173` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:11.918052 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f53175-0dec-4170-bb52-7ae5aa9f703d None admin] RBAC: Authorizing `identity:get_user(user_id=f720a2ce258149afacb65b2c45fa2173)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:11.919903 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f53175-0dec-4170-bb52-7ae5aa9f703d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:11.922177 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 83/171] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:13:11 2022] GET /identity/v3/users/f720a2ce258149afacb65b2c45fa2173 => generated 275 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:11.928792 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f53175-0dec-4170-bb52-7ae5aa9f703d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:11.981342 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23dfa61-b090-4e1c-bf37-31a77ef39f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:11.982367 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23dfa61-b090-4e1c-bf37-31a77ef39f8d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:11.982787 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23dfa61-b090-4e1c-bf37-31a77ef39f8d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:11.983227 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23dfa61-b090-4e1c-bf37-31a77ef39f8d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:12.002774 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23dfa61-b090-4e1c-bf37-31a77ef39f8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:12.008091 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-f23dfa61-b090-4e1c-bf37-31a77ef39f8d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 10:13:12.008091 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:12.008091 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:12.008091 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:12.008091 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:12.008091 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:12.008091 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:12.008091 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:12.008091 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:12.008091 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:12.008091 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:12.008091 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:12.008091 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:12.008091 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:12.008091 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:12.008091 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:12.009073 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:12.009073 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:12.009073 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:12.009073 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:12.009073 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:12.009073 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:12.009073 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:12.009073 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:12.009073 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:12.009073 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:12.009073 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:12.009073 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:12.009073 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:12.009073 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:12.009073 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:12.009073 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:12.009954 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:12.009954 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:12.009954 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:12.009954 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:12.009954 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:12.009954 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:12.009954 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:12.009954 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:12.009954 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:12.009954 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:12.009954 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 10:13:12.009954 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:13:12.012081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 89/172] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:13:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:13:12.018053 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23dfa61-b090-4e1c-bf37-31a77ef39f8d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:12.093320 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:12.120079 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:12.173444 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f53175-0dec-4170-bb52-7ae5aa9f703d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:12.211206 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23dfa61-b090-4e1c-bf37-31a77ef39f8d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:12.246608 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa2c10f-8b4f-4d0e-8c75-ad681fad5e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:12.247322 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2c10f-8b4f-4d0e-8c75-ad681fad5e75 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:12.247570 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2c10f-8b4f-4d0e-8c75-ad681fad5e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:12.248313 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2c10f-8b4f-4d0e-8c75-ad681fad5e75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:12.282769 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b48f602-1236-4cb7-8864-8beff04636b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:12.283418 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b48f602-1236-4cb7-8864-8beff04636b6 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:12.283702 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b48f602-1236-4cb7-8864-8beff04636b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:12.284028 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b48f602-1236-4cb7-8864-8beff04636b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:12.284838 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b48f602-1236-4cb7-8864-8beff04636b6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:12.285164 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b48f602-1236-4cb7-8864-8beff04636b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:12.286553 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b48f602-1236-4cb7-8864-8beff04636b6 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:12.302212 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 90/173] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 10:13:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:13:12.303563 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da52666-c10d-4240-ba1a-44e35e31bead None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:12.303858 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da52666-c10d-4240-ba1a-44e35e31bead None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:12.304135 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da52666-c10d-4240-ba1a-44e35e31bead None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:12.305562 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 91/174] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:12.310850 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da52666-c10d-4240-ba1a-44e35e31bead None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:12.359619 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa2c10f-8b4f-4d0e-8c75-ad681fad5e75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:12.361140 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa2c10f-8b4f-4d0e-8c75-ad681fad5e75 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:12.419560 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da52666-c10d-4240-ba1a-44e35e31bead None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:12.438547 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:12.459515 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79215f95-c4b2-44ef-82f5-d3fa20f16c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:12.460174 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79215f95-c4b2-44ef-82f5-d3fa20f16c63 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:12.460452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79215f95-c4b2-44ef-82f5-d3fa20f16c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:12.460726 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79215f95-c4b2-44ef-82f5-d3fa20f16c63 None admin] PATH_INFO: `/v3/users/f720a2ce258149afacb65b2c45fa2173` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:12.462407 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79215f95-c4b2-44ef-82f5-d3fa20f16c63 None admin] RBAC: Authorizing `identity:get_user(user_id=f720a2ce258149afacb65b2c45fa2173)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:12.465534 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79215f95-c4b2-44ef-82f5-d3fa20f16c63 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:12.467740 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 92/175] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:13:12 2022] GET /identity/v3/users/f720a2ce258149afacb65b2c45fa2173 => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:13:12.469568 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4105c1-30d8-44f6-abb1-aad4c44a853d None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:12.469845 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4105c1-30d8-44f6-abb1-aad4c44a853d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:12.470118 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4105c1-30d8-44f6-abb1-aad4c44a853d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:12.492232 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-7d4105c1-30d8-44f6-abb1-aad4c44a853d None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:12.493823 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7d4105c1-30d8-44f6-abb1-aad4c44a853d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:12.495339 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:12.523857 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 84/176] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:13:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 10:13:12.525099 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80b290-ed6d-40a8-88fc-0e7516882c7e None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:12.525380 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80b290-ed6d-40a8-88fc-0e7516882c7e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:12.525653 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80b290-ed6d-40a8-88fc-0e7516882c7e None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:12.526967 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 85/177] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:12.536938 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80b290-ed6d-40a8-88fc-0e7516882c7e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:12.569316 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 93/178] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:12.575410 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4105c1-30d8-44f6-abb1-aad4c44a853d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:12.675190 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80b290-ed6d-40a8-88fc-0e7516882c7e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:12.718792 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac081fb-e28a-47ee-96c6-0037b602e8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:12.719461 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac081fb-e28a-47ee-96c6-0037b602e8b3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:12.719906 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac081fb-e28a-47ee-96c6-0037b602e8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:12.720185 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac081fb-e28a-47ee-96c6-0037b602e8b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:12.726794 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4105c1-30d8-44f6-abb1-aad4c44a853d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:12.745231 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:12.763868 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d5e420-ce61-41be-90c8-4456ee007263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:12.764522 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d5e420-ce61-41be-90c8-4456ee007263 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:12.764812 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d5e420-ce61-41be-90c8-4456ee007263 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:12.765185 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d5e420-ce61-41be-90c8-4456ee007263 None admin] PATH_INFO: `/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:12.766951 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d5e420-ce61-41be-90c8-4456ee007263 None admin] RBAC: Authorizing `identity:get_project(project_id=a9d326f0c0fa4a38aa7ba8c8c6797c5a)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:12.767178 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:12.768898 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d5e420-ce61-41be-90c8-4456ee007263 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:12.771007 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 94/179] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:13:12 2022] GET /identity/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:12.776151 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6edf0-5d08-4dae-9f5b-893ac6e72257 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:12.776440 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6edf0-5d08-4dae-9f5b-893ac6e72257 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:12.776716 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6edf0-5d08-4dae-9f5b-893ac6e72257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:12.794436 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac081fb-e28a-47ee-96c6-0037b602e8b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:12.796390 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac081fb-e28a-47ee-96c6-0037b602e8b3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:12.801249 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-55c6edf0-5d08-4dae-9f5b-893ac6e72257 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:12.802435 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-55c6edf0-5d08-4dae-9f5b-893ac6e72257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:12.875032 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 95/180] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:12.875912 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c6edf0-5d08-4dae-9f5b-893ac6e72257 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:12.905512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 86/181] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:13:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 10:13:12.905512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b5687-ff14-4103-b2ef-7ef8a7f90c7b None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:12.905512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b5687-ff14-4103-b2ef-7ef8a7f90c7b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:12.905512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b5687-ff14-4103-b2ef-7ef8a7f90c7b None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:12.905512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 87/182] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:12.905512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b5687-ff14-4103-b2ef-7ef8a7f90c7b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:12.980712 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:12.983089 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c6edf0-5d08-4dae-9f5b-893ac6e72257 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:12.997075 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:13.007888 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b5687-ff14-4103-b2ef-7ef8a7f90c7b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:13.017691 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39bbe36-53fe-438a-aaeb-f47255a2422b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:13.018343 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bbe36-53fe-438a-aaeb-f47255a2422b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:13.018621 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bbe36-53fe-438a-aaeb-f47255a2422b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:13.018892 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bbe36-53fe-438a-aaeb-f47255a2422b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:13.028965 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39bbe36-53fe-438a-aaeb-f47255a2422b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:13.034044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-b39bbe36-53fe-438a-aaeb-f47255a2422b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:13:13.034044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:13.034044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:13.034044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:13.034044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:13.034044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:13.034044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:13.034044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:13.034044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:13.034044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:13.034044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:13.034044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:13.034044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:13.034044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:13.034044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:13.034044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:13.034994 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:13.034994 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:13.034994 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:13.034994 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:13.034994 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:13.034994 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:13.034994 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:13.034994 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:13.034994 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:13.034994 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:13.034994 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:13.034994 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:13.034994 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:13.034994 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:13.034994 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:13.034994 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:13.035776 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:13.035776 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:13.035776 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:13.035776 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:13.035776 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:13.035776 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:13.035776 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:13.035776 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:13.035776 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:13.035776 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:13.035776 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:13:13.035776 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:13:13.035776 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 96/183] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 10:13:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:13.036418 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fe01d-54b9-4577-802f-f7144cad1364 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:13.036690 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fe01d-54b9-4577-802f-f7144cad1364 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:13.036953 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fe01d-54b9-4577-802f-f7144cad1364 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:13.038195 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 97/184] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:13.042516 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5fe01d-54b9-4577-802f-f7144cad1364 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:13.046606 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28177123-ec1e-482f-9d55-09207d94775a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:13.047281 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28177123-ec1e-482f-9d55-09207d94775a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:13.047558 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28177123-ec1e-482f-9d55-09207d94775a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:13.047829 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28177123-ec1e-482f-9d55-09207d94775a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:13.048617 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28177123-ec1e-482f-9d55-09207d94775a None admin] RBAC: Adding query filter params (role.id=57a070b43ad940f1886c69721dd304c2, scope.project.id=a9d326f0c0fa4a38aa7ba8c8c6797c5a, user.id=f720a2ce258149afacb65b2c45fa2173) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:13.048922 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28177123-ec1e-482f-9d55-09207d94775a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:13.050368 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28177123-ec1e-482f-9d55-09207d94775a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:13.056676 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 88/185] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:13:12 2022] GET /identity/v3/role_assignments?user.id=f720a2ce258149afacb65b2c45fa2173&scope.project.id=a9d326f0c0fa4a38aa7ba8c8c6797c5a&role.id=57a070b43ad940f1886c69721dd304c2 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:13.057449 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28177123-ec1e-482f-9d55-09207d94775a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:13.169300 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28177123-ec1e-482f-9d55-09207d94775a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:13.185867 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5fe01d-54b9-4577-802f-f7144cad1364 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:13.206886 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ba5648-5715-40ef-9525-60bfefdcd6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:13.207582 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba5648-5715-40ef-9525-60bfefdcd6bd None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:13.207861 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba5648-5715-40ef-9525-60bfefdcd6bd None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:13.208130 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba5648-5715-40ef-9525-60bfefdcd6bd None admin] PATH_INFO: `/v3/projects/387614ebbd51447b89e7a0c2ee6d4b96` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:13.209320 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ba5648-5715-40ef-9525-60bfefdcd6bd None admin] RBAC: Authorizing `identity:get_project(project_id=387614ebbd51447b89e7a0c2ee6d4b96)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:13.210804 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ba5648-5715-40ef-9525-60bfefdcd6bd None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:13.217367 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 89/186] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:13:13 2022] GET /identity/v3/projects/387614ebbd51447b89e7a0c2ee6d4b96 => generated 312 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:13.217367 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839fb7b8-2ef2-454e-aa47-b1875a3f8947 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:13.217367 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839fb7b8-2ef2-454e-aa47-b1875a3f8947 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:13.217367 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839fb7b8-2ef2-454e-aa47-b1875a3f8947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:13.228297 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:13.235414 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-839fb7b8-2ef2-454e-aa47-b1875a3f8947 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:13.236684 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-839fb7b8-2ef2-454e-aa47-b1875a3f8947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:13.239403 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e567e6-e29a-473b-b85a-f2a035eb1407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:13.240399 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e567e6-e29a-473b-b85a-f2a035eb1407 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:13.240399 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e567e6-e29a-473b-b85a-f2a035eb1407 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:13.240606 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e567e6-e29a-473b-b85a-f2a035eb1407 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:13.241765 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e567e6-e29a-473b-b85a-f2a035eb1407 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:13.243265 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e567e6-e29a-473b-b85a-f2a035eb1407 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:13.247391 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 98/187] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:13:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:13.247391 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e567e6-e29a-473b-b85a-f2a035eb1407 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:13.294695 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:13.314102 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 90/188] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:13.315326 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e252f7-f21c-4963-b55e-9f82ee5b2572 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:13.315606 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e252f7-f21c-4963-b55e-9f82ee5b2572 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:13.315875 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e252f7-f21c-4963-b55e-9f82ee5b2572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:13.333686 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-13e252f7-f21c-4963-b55e-9f82ee5b2572 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:13.334858 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-13e252f7-f21c-4963-b55e-9f82ee5b2572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:13.353435 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e567e6-e29a-473b-b85a-f2a035eb1407 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:13.386739 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc21014f-4c9c-48ee-ba6c-52806bd5329d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:13.387618 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21014f-4c9c-48ee-ba6c-52806bd5329d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:13.387889 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21014f-4c9c-48ee-ba6c-52806bd5329d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:13.388151 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21014f-4c9c-48ee-ba6c-52806bd5329d None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:13.388876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc21014f-4c9c-48ee-ba6c-52806bd5329d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:13.389171 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc21014f-4c9c-48ee-ba6c-52806bd5329d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:13.390631 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc21014f-4c9c-48ee-ba6c-52806bd5329d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:13.398446 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 99/189] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:13:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:13.403009 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc21014f-4c9c-48ee-ba6c-52806bd5329d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:13.404265 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 91/190] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:13.405070 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e252f7-f21c-4963-b55e-9f82ee5b2572 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:13.467989 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:13.502163 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e252f7-f21c-4963-b55e-9f82ee5b2572 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:13.503786 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc21014f-4c9c-48ee-ba6c-52806bd5329d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:13.506081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:13.535378 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92444c5-3038-4cf1-84b6-071c8942c642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:13.536010 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92444c5-3038-4cf1-84b6-071c8942c642 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:13.536289 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92444c5-3038-4cf1-84b6-071c8942c642 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:13.536562 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92444c5-3038-4cf1-84b6-071c8942c642 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:13.537248 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92444c5-3038-4cf1-84b6-071c8942c642 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:13.538453 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019fd85b-220a-47b5-995c-496e468b3726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:13.538732 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92444c5-3038-4cf1-84b6-071c8942c642 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:13.539089 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019fd85b-220a-47b5-995c-496e468b3726 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:13.539368 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019fd85b-220a-47b5-995c-496e468b3726 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:13.539633 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019fd85b-220a-47b5-995c-496e468b3726 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:13.540418 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019fd85b-220a-47b5-995c-496e468b3726 None admin] RBAC: Adding query filter params (role.id=c4ba94fa4bd34a09ba87d357142d9db8, scope.project.id=387614ebbd51447b89e7a0c2ee6d4b96, user.id=f720a2ce258149afacb65b2c45fa2173) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:13.540723 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019fd85b-220a-47b5-995c-496e468b3726 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:13.542140 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019fd85b-220a-47b5-995c-496e468b3726 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:13.548625 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 100/191] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:13:13 2022] GET /identity/v3/role_assignments?user.id=f720a2ce258149afacb65b2c45fa2173&scope.project.id=387614ebbd51447b89e7a0c2ee6d4b96&role.id=c4ba94fa4bd34a09ba87d357142d9db8 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:13.549495 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019fd85b-220a-47b5-995c-496e468b3726 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:13.571533 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 92/192] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 5 10:13:13 2022] POST /identity/v3/users => generated 297 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:13:13.571533 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92444c5-3038-4cf1-84b6-071c8942c642 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:13.689820 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019fd85b-220a-47b5-995c-496e468b3726 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:13.701894 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92444c5-3038-4cf1-84b6-071c8942c642 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:13.731301 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:13.741084 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d69009b-8c59-4299-95aa-abd0820ccc36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:13.742114 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d69009b-8c59-4299-95aa-abd0820ccc36 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:13.742493 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d69009b-8c59-4299-95aa-abd0820ccc36 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:13.742865 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d69009b-8c59-4299-95aa-abd0820ccc36 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:13.754344 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7021e5-fbb2-4e2d-ab61-cc027bf7548b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:13.755033 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7021e5-fbb2-4e2d-ab61-cc027bf7548b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:13.755311 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7021e5-fbb2-4e2d-ab61-cc027bf7548b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:13.755579 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7021e5-fbb2-4e2d-ab61-cc027bf7548b None admin] PATH_INFO: `/v3/users/6e6904b0179c4a96bafa44a2c486016f` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:13.757456 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d69009b-8c59-4299-95aa-abd0820ccc36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:13.765916 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-6d69009b-8c59-4299-95aa-abd0820ccc36 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 10:13:13.765916 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:13.765916 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:13.765916 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:13.765916 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:13.765916 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:13.765916 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:13.765916 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:13.765916 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:13.765916 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:13.765916 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:13.765916 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:13.765916 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:13.765916 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:13.765916 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:13.765916 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:13.766844 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:13.766844 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:13.766844 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:13.766844 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:13.766844 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:13.766844 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:13.766844 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:13.766844 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:13.766844 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:13.766844 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:13.766844 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:13.766844 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:13.766844 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:13.766844 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:13.766844 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:13.766844 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:13.768138 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:13.768138 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:13.768138 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:13.768138 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:13.768138 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:13.768138 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:13.768138 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:13.768138 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:13.768138 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:13.768138 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:13.768138 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 10:13:13.768138 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:13:13.768138 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 101/193] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:13:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:13.768138 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d69009b-8c59-4299-95aa-abd0820ccc36 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:13.775694 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7021e5-fbb2-4e2d-ab61-cc027bf7548b None admin] RBAC: Authorizing `identity:get_user(user_id=6e6904b0179c4a96bafa44a2c486016f)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:13.777271 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7021e5-fbb2-4e2d-ab61-cc027bf7548b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:13.780151 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 93/194] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:13:13 2022] GET /identity/v3/users/6e6904b0179c4a96bafa44a2c486016f => generated 283 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:13.781073 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7021e5-fbb2-4e2d-ab61-cc027bf7548b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:13.808050 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:13.912950 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d69009b-8c59-4299-95aa-abd0820ccc36 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:13.927469 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7021e5-fbb2-4e2d-ab61-cc027bf7548b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:13.961699 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e00d9df-d995-48a2-8231-6d095e6d0ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:13.962598 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00d9df-d995-48a2-8231-6d095e6d0ae3 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:13.963049 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00d9df-d995-48a2-8231-6d095e6d0ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:13.963454 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00d9df-d995-48a2-8231-6d095e6d0ae3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:13.971272 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c27ad0e-97f1-4c6a-97af-735deb5cce57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:13.971986 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c27ad0e-97f1-4c6a-97af-735deb5cce57 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:13.972330 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c27ad0e-97f1-4c6a-97af-735deb5cce57 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:13.972667 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c27ad0e-97f1-4c6a-97af-735deb5cce57 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:13.973459 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c27ad0e-97f1-4c6a-97af-735deb5cce57 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:13.973835 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c27ad0e-97f1-4c6a-97af-735deb5cce57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:13.975319 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c27ad0e-97f1-4c6a-97af-735deb5cce57 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:13.976055 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e00d9df-d995-48a2-8231-6d095e6d0ae3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:13.981636 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-7e00d9df-d995-48a2-8231-6d095e6d0ae3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 10:13:13.981636 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:13.981636 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:13.981636 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:13.981636 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:13.981636 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:13.981636 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:13.981636 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:13.981636 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:13.981636 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:13.981636 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:13.981636 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:13.981636 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:13.981636 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:13.981636 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:13.981636 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:13.982736 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:13.982736 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:13.982736 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:13.982736 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:13.982736 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:13.982736 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:13.982736 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:13.982736 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:13.982736 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:13.982736 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:13.982736 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:13.982736 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:13.982736 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:13.982736 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:13.982736 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:13.982736 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:13.983605 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:13.983605 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:13.983605 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:13.983605 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:13.983605 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:13.983605 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:13.983605 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:13.983605 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:13.983605 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:13.983605 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:13.983605 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 10:13:13.983605 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:13:13.987090 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 102/195] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 5 10:13:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:13.987422 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 94/196] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 10:13:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:13.988354 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e00d9df-d995-48a2-8231-6d095e6d0ae3 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:13.993667 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c27ad0e-97f1-4c6a-97af-735deb5cce57 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:14.056668 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:14.090699 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:14.113005 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e00d9df-d995-48a2-8231-6d095e6d0ae3 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:14.132837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c27ad0e-97f1-4c6a-97af-735deb5cce57 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:14.157773 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55e221-8b0f-48c4-b5e0-1ba37ca65bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:14.158800 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55e221-8b0f-48c4-b5e0-1ba37ca65bcd None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:14.159303 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55e221-8b0f-48c4-b5e0-1ba37ca65bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:14.159867 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55e221-8b0f-48c4-b5e0-1ba37ca65bcd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:14.174655 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2d3106-94d4-40b9-b26e-f5c4b21fe13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:14.175268 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d3106-94d4-40b9-b26e-f5c4b21fe13c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:14.175898 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d3106-94d4-40b9-b26e-f5c4b21fe13c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:14.175898 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d3106-94d4-40b9-b26e-f5c4b21fe13c None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:14.176643 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2d3106-94d4-40b9-b26e-f5c4b21fe13c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:14.176874 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2d3106-94d4-40b9-b26e-f5c4b21fe13c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:14.178316 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2d3106-94d4-40b9-b26e-f5c4b21fe13c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:14.187305 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 95/197] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:13:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:14.188054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2d3106-94d4-40b9-b26e-f5c4b21fe13c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:14.265033 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c55e221-8b0f-48c4-b5e0-1ba37ca65bcd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:14.267024 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c55e221-8b0f-48c4-b5e0-1ba37ca65bcd None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:14.306380 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2d3106-94d4-40b9-b26e-f5c4b21fe13c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:14.309082 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:14.356928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af946141-159b-40b5-8cf5-3cfa7354ecf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:14.357724 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af946141-159b-40b5-8cf5-3cfa7354ecf2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:14.357917 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af946141-159b-40b5-8cf5-3cfa7354ecf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:14.358222 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af946141-159b-40b5-8cf5-3cfa7354ecf2 None admin] PATH_INFO: `/v3/users/dd861837dc1d4bb2a5bcc19c545b8432` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:14.359669 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af946141-159b-40b5-8cf5-3cfa7354ecf2 None admin] RBAC: Authorizing `identity:get_user(user_id=dd861837dc1d4bb2a5bcc19c545b8432)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:14.361064 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af946141-159b-40b5-8cf5-3cfa7354ecf2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:14.362750 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 96/198] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:13:14 2022] GET /identity/v3/users/dd861837dc1d4bb2a5bcc19c545b8432 => generated 289 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:14.364477 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af946141-159b-40b5-8cf5-3cfa7354ecf2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:14.498370 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:14.518666 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af946141-159b-40b5-8cf5-3cfa7354ecf2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:14.546020 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 103/199] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:13:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 10:13:14.551806 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55e221-8b0f-48c4-b5e0-1ba37ca65bcd None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:14.561275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37923ffd-64d2-4ee8-9757-23b0153b91de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:14.562207 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37923ffd-64d2-4ee8-9757-23b0153b91de None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:14.562631 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37923ffd-64d2-4ee8-9757-23b0153b91de None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:14.563106 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37923ffd-64d2-4ee8-9757-23b0153b91de None admin] PATH_INFO: `/v3/users/6e6904b0179c4a96bafa44a2c486016f` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:14.564494 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37923ffd-64d2-4ee8-9757-23b0153b91de None admin] RBAC: Authorizing `identity:get_user(user_id=6e6904b0179c4a96bafa44a2c486016f)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:14.566101 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37923ffd-64d2-4ee8-9757-23b0153b91de None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:14.567887 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 97/200] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:13:14 2022] GET /identity/v3/users/6e6904b0179c4a96bafa44a2c486016f => generated 283 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:14.568936 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37923ffd-64d2-4ee8-9757-23b0153b91de None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:14.637042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:14.706087 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37923ffd-64d2-4ee8-9757-23b0153b91de None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:14.743804 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55e221-8b0f-48c4-b5e0-1ba37ca65bcd None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:14.753123 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e64b4e-b378-4ae9-be5f-02bbafa69578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:14.754031 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e64b4e-b378-4ae9-be5f-02bbafa69578 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:14.754877 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e64b4e-b378-4ae9-be5f-02bbafa69578 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:14.755383 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e64b4e-b378-4ae9-be5f-02bbafa69578 None admin] PATH_INFO: `/v3/projects/400a983b4cd947d487b7fa86a5f5c339` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:14.757483 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e64b4e-b378-4ae9-be5f-02bbafa69578 None admin] RBAC: Authorizing `identity:get_project(project_id=400a983b4cd947d487b7fa86a5f5c339)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:14.759952 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e64b4e-b378-4ae9-be5f-02bbafa69578 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:14.762317 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 98/201] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:13:14 2022] GET /identity/v3/projects/400a983b4cd947d487b7fa86a5f5c339 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:14.763297 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e64b4e-b378-4ae9-be5f-02bbafa69578 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:14.785360 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe042c01-7d4e-437f-a045-9ddb674c423a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:14.785987 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe042c01-7d4e-437f-a045-9ddb674c423a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:14.786265 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe042c01-7d4e-437f-a045-9ddb674c423a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:14.786534 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe042c01-7d4e-437f-a045-9ddb674c423a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:14.848391 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:14.864313 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe042c01-7d4e-437f-a045-9ddb674c423a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:14.866001 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe042c01-7d4e-437f-a045-9ddb674c423a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:14.906189 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e64b4e-b378-4ae9-be5f-02bbafa69578 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:14.927387 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:14.953845 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6be8870-7ce5-4a63-91b4-26372377fa26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:14.954590 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be8870-7ce5-4a63-91b4-26372377fa26 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:14.955040 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be8870-7ce5-4a63-91b4-26372377fa26 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:14.955383 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be8870-7ce5-4a63-91b4-26372377fa26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:14.991323 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 104/202] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:13:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 10:13:14.991323 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe042c01-7d4e-437f-a045-9ddb674c423a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:15.046479 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6be8870-7ce5-4a63-91b4-26372377fa26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:15.049271 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6be8870-7ce5-4a63-91b4-26372377fa26 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:15.089315 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:15.107491 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe042c01-7d4e-437f-a045-9ddb674c423a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:15.160069 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 99/203] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:13:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 10:13:15.160856 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ff947a-a4e2-4deb-8786-b6972d6f8c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:15.161530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6be8870-7ce5-4a63-91b4-26372377fa26 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:15.162123 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff947a-a4e2-4deb-8786-b6972d6f8c4c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:15.162769 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff947a-a4e2-4deb-8786-b6972d6f8c4c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:15.163282 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff947a-a4e2-4deb-8786-b6972d6f8c4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:15.164612 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ff947a-a4e2-4deb-8786-b6972d6f8c4c None admin] RBAC: Adding query filter params (role.id=daef89adcaeb48a382a0bcb03340ccbb, scope.project.id=400a983b4cd947d487b7fa86a5f5c339, user.id=6e6904b0179c4a96bafa44a2c486016f) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:15.165194 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ff947a-a4e2-4deb-8786-b6972d6f8c4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:15.167667 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ff947a-a4e2-4deb-8786-b6972d6f8c4c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:15.175197 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:15.179340 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 105/204] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:13:14 2022] GET /identity/v3/role_assignments?user.id=6e6904b0179c4a96bafa44a2c486016f&scope.project.id=400a983b4cd947d487b7fa86a5f5c339&role.id=daef89adcaeb48a382a0bcb03340ccbb => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:15.180743 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ff947a-a4e2-4deb-8786-b6972d6f8c4c None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:15.285768 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6be8870-7ce5-4a63-91b4-26372377fa26 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:15.329637 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b10a0-d979-4212-8e91-dd13e04ad01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:15.331287 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b10a0-d979-4212-8e91-dd13e04ad01d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:15.331777 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b10a0-d979-4212-8e91-dd13e04ad01d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:15.332256 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b10a0-d979-4212-8e91-dd13e04ad01d None admin] PATH_INFO: `/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:15.333815 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b10a0-d979-4212-8e91-dd13e04ad01d None admin] RBAC: Authorizing `identity:get_project(project_id=a9d326f0c0fa4a38aa7ba8c8c6797c5a)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:15.335847 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b10a0-d979-4212-8e91-dd13e04ad01d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:15.337939 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 100/205] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:13:15 2022] GET /identity/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:15.344308 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b10a0-d979-4212-8e91-dd13e04ad01d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:15.441225 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:15.460867 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ff947a-a4e2-4deb-8786-b6972d6f8c4c None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:15.470429 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b10a0-d979-4212-8e91-dd13e04ad01d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:15.509511 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6203cf6a-4a34-4bc3-82cf-adbb7afdd5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:15.510393 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203cf6a-4a34-4bc3-82cf-adbb7afdd5e5 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:15.510817 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203cf6a-4a34-4bc3-82cf-adbb7afdd5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:15.511795 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203cf6a-4a34-4bc3-82cf-adbb7afdd5e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:15.513050 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6203cf6a-4a34-4bc3-82cf-adbb7afdd5e5 None admin] RBAC: Adding query filter params (role.id=cf5de414a035475b8b75626cb19a2de2, scope.project.id=a9d326f0c0fa4a38aa7ba8c8c6797c5a, user.id=dd861837dc1d4bb2a5bcc19c545b8432) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:15.513512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6203cf6a-4a34-4bc3-82cf-adbb7afdd5e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:15.515876 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6203cf6a-4a34-4bc3-82cf-adbb7afdd5e5 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:15.524754 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 101/206] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:13:15 2022] GET /identity/v3/role_assignments?user.id=dd861837dc1d4bb2a5bcc19c545b8432&scope.project.id=a9d326f0c0fa4a38aa7ba8c8c6797c5a&role.id=cf5de414a035475b8b75626cb19a2de2 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:15.531000 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01347f48-b907-494b-a255-8f3b6d77f5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:15.531000 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01347f48-b907-494b-a255-8f3b6d77f5bd None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:15.531000 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01347f48-b907-494b-a255-8f3b6d77f5bd None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:15.531000 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01347f48-b907-494b-a255-8f3b6d77f5bd None admin] PATH_INFO: `/v3/projects/400a983b4cd947d487b7fa86a5f5c339` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:15.531821 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01347f48-b907-494b-a255-8f3b6d77f5bd None admin] RBAC: Authorizing `identity:get_project(project_id=400a983b4cd947d487b7fa86a5f5c339)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:15.531821 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01347f48-b907-494b-a255-8f3b6d77f5bd None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:15.533043 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 106/207] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:13:15 2022] GET /identity/v3/projects/400a983b4cd947d487b7fa86a5f5c339 => generated 302 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:15.539317 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6203cf6a-4a34-4bc3-82cf-adbb7afdd5e5 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:15.650714 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6203cf6a-4a34-4bc3-82cf-adbb7afdd5e5 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:15.688646 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98acec7a-f34d-40d8-8ac6-4460dcb24626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:15.689280 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98acec7a-f34d-40d8-8ac6-4460dcb24626 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:15.689557 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98acec7a-f34d-40d8-8ac6-4460dcb24626 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:15.689827 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98acec7a-f34d-40d8-8ac6-4460dcb24626 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:15.690621 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98acec7a-f34d-40d8-8ac6-4460dcb24626 None admin] RBAC: Adding query filter params (role.id=57a070b43ad940f1886c69721dd304c2, scope.project.id=400a983b4cd947d487b7fa86a5f5c339, user.id=6e6904b0179c4a96bafa44a2c486016f) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:15.690944 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98acec7a-f34d-40d8-8ac6-4460dcb24626 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:15.692399 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98acec7a-f34d-40d8-8ac6-4460dcb24626 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:15.698621 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 102/208] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:13:15 2022] GET /identity/v3/role_assignments?user.id=6e6904b0179c4a96bafa44a2c486016f&scope.project.id=400a983b4cd947d487b7fa86a5f5c339&role.id=57a070b43ad940f1886c69721dd304c2 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:15.814336 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ba242-4ab0-4016-8235-b1ededcdc138 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:15.814663 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ba242-4ab0-4016-8235-b1ededcdc138 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:15.814949 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ba242-4ab0-4016-8235-b1ededcdc138 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:15.816411 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 107/209] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:15.822279 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41436fce-1799-444f-9ac0-f52c00e43d1b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:15.822594 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41436fce-1799-444f-9ac0-f52c00e43d1b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:15.822825 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41436fce-1799-444f-9ac0-f52c00e43d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:15.831611 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:15.844065 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-41436fce-1799-444f-9ac0-f52c00e43d1b None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:15.845269 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-41436fce-1799-444f-9ac0-f52c00e43d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:15.923098 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 103/210] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 10:13:15.941601 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51ba242-4ab0-4016-8235-b1ededcdc138 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:15.965151 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:16.042938 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c32cc5f-797d-4ec5-a7a4-3d3b1e44c73b None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:16.043235 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c32cc5f-797d-4ec5-a7a4-3d3b1e44c73b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:16.043504 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c32cc5f-797d-4ec5-a7a4-3d3b1e44c73b None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:16.045381 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 104/211] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 10:13:16.050735 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a3b8e2-8f04-4c7e-a28f-578f8ed4a28f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:16.051307 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a3b8e2-8f04-4c7e-a28f-578f8ed4a28f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:16.051885 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a3b8e2-8f04-4c7e-a28f-578f8ed4a28f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:16.052250 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51ba242-4ab0-4016-8235-b1ededcdc138 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:16.072290 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c7a3b8e2-8f04-4c7e-a28f-578f8ed4a28f None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:16.073804 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c7a3b8e2-8f04-4c7e-a28f-578f8ed4a28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:16.092951 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1847ed07-bc67-4b37-846b-a0b8bee66d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:16.094064 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847ed07-bc67-4b37-846b-a0b8bee66d90 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:16.094692 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847ed07-bc67-4b37-846b-a0b8bee66d90 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:16.095232 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847ed07-bc67-4b37-846b-a0b8bee66d90 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:16.107700 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1847ed07-bc67-4b37-846b-a0b8bee66d90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:16.113796 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-1847ed07-bc67-4b37-846b-a0b8bee66d90 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 10:13:16.113796 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:16.113796 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:16.113796 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:16.113796 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:16.113796 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:16.113796 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:16.113796 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:16.113796 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:16.113796 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:16.113796 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:16.113796 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:16.113796 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:16.113796 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:16.113796 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:16.113796 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:16.114668 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:16.114668 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:16.114668 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:16.114668 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:16.114668 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:16.114668 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:16.114668 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:16.114668 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:16.114668 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:16.114668 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:16.114668 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:16.114668 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:16.114668 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:16.114668 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:16.114668 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:16.114668 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:16.115489 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:16.115489 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:16.115489 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:16.115489 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:16.115489 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:16.115489 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:16.115489 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:16.115489 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:16.115489 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:16.115489 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:16.115489 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 10:13:16.115489 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:13:16.118682 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 108/212] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:13:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:16.124794 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1847ed07-bc67-4b37-846b-a0b8bee66d90 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:16.168166 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 105/213] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 10:13:16.194887 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a3b8e2-8f04-4c7e-a28f-578f8ed4a28f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:16.214462 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:16.235357 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1847ed07-bc67-4b37-846b-a0b8bee66d90 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:16.261751 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:16.269876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326a49c8-a556-4d9c-8f2d-0d31407f1a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:16.270502 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a49c8-a556-4d9c-8f2d-0d31407f1a19 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:16.270780 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a49c8-a556-4d9c-8f2d-0d31407f1a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:16.271082 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a49c8-a556-4d9c-8f2d-0d31407f1a19 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:16.271983 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326a49c8-a556-4d9c-8f2d-0d31407f1a19 None admin] RBAC: Adding query filter params (name=member) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:16.272288 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326a49c8-a556-4d9c-8f2d-0d31407f1a19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:16.273693 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326a49c8-a556-4d9c-8f2d-0d31407f1a19 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:16.282489 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 109/214] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 10:13:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:16.287757 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326a49c8-a556-4d9c-8f2d-0d31407f1a19 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:16.295804 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a3b8e2-8f04-4c7e-a28f-578f8ed4a28f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:16.330110 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d8605-b070-4b6a-b68c-ca67a35d4873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:16.330758 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d8605-b070-4b6a-b68c-ca67a35d4873 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:16.331084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d8605-b070-4b6a-b68c-ca67a35d4873 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:16.331361 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d8605-b070-4b6a-b68c-ca67a35d4873 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:16.332496 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776d8605-b070-4b6a-b68c-ca67a35d4873 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:16.333946 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776d8605-b070-4b6a-b68c-ca67a35d4873 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:16.335816 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 106/215] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:13:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:16.341182 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d8605-b070-4b6a-b68c-ca67a35d4873 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:16.390207 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326a49c8-a556-4d9c-8f2d-0d31407f1a19 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:16.426347 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558140ff-f54c-4940-97de-a922b837b7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:16.427030 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558140ff-f54c-4940-97de-a922b837b7e6 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:16.427314 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558140ff-f54c-4940-97de-a922b837b7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:16.427589 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558140ff-f54c-4940-97de-a922b837b7e6 None admin] PATH_INFO: `/v3/users/b17ad37d9c5d445990fce41f2172124d` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:16.439625 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d8605-b070-4b6a-b68c-ca67a35d4873 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:16.443306 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:16.446826 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558140ff-f54c-4940-97de-a922b837b7e6 None admin] RBAC: Authorizing `identity:get_user(user_id=b17ad37d9c5d445990fce41f2172124d)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:16.448914 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558140ff-f54c-4940-97de-a922b837b7e6 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:16.456455 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 110/216] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:13:16 2022] GET /identity/v3/users/b17ad37d9c5d445990fce41f2172124d => generated 297 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:13:16.457756 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558140ff-f54c-4940-97de-a922b837b7e6 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:16.479368 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:16.481270 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a06528-cbfe-44d2-9c08-344b2cd4314a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:16.481937 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a06528-cbfe-44d2-9c08-344b2cd4314a None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:16.482217 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a06528-cbfe-44d2-9c08-344b2cd4314a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:16.482610 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a06528-cbfe-44d2-9c08-344b2cd4314a None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:16.483345 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a06528-cbfe-44d2-9c08-344b2cd4314a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:16.485204 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a06528-cbfe-44d2-9c08-344b2cd4314a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:16.513507 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 107/217] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 5 10:13:16 2022] POST /identity/v3/users => generated 297 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:13:16.592832 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558140ff-f54c-4940-97de-a922b837b7e6 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:16.631547 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd48c1c-d3b9-4d67-9ce3-365003a4ad2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:16.632245 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd48c1c-d3b9-4d67-9ce3-365003a4ad2d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:16.632527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd48c1c-d3b9-4d67-9ce3-365003a4ad2d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:16.632798 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd48c1c-d3b9-4d67-9ce3-365003a4ad2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:16.706003 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:16.709176 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd48c1c-d3b9-4d67-9ce3-365003a4ad2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:16.710678 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd48c1c-d3b9-4d67-9ce3-365003a4ad2d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:16.813795 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 111/218] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:13:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 10:13:16.820040 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a06528-cbfe-44d2-9c08-344b2cd4314a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:16.953342 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a06528-cbfe-44d2-9c08-344b2cd4314a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:16.987469 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba659f67-c9d2-48c9-99af-1d0b7df6fa2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:16.988406 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba659f67-c9d2-48c9-99af-1d0b7df6fa2e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:16.988823 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba659f67-c9d2-48c9-99af-1d0b7df6fa2e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:16.989232 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba659f67-c9d2-48c9-99af-1d0b7df6fa2e None admin] PATH_INFO: `/v3/projects/400a983b4cd947d487b7fa86a5f5c339` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:16.990538 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba659f67-c9d2-48c9-99af-1d0b7df6fa2e None admin] RBAC: Authorizing `identity:get_project(project_id=400a983b4cd947d487b7fa86a5f5c339)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:16.992212 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba659f67-c9d2-48c9-99af-1d0b7df6fa2e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:16.994055 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 108/219] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:13:16 2022] GET /identity/v3/projects/400a983b4cd947d487b7fa86a5f5c339 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:17.000018 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd48c1c-d3b9-4d67-9ce3-365003a4ad2d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:17.144081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:17.153315 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd48c1c-d3b9-4d67-9ce3-365003a4ad2d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:17.199328 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c22338-ce85-418a-b7f7-8a033548b6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:17.199959 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c22338-ce85-418a-b7f7-8a033548b6f2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:17.200242 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c22338-ce85-418a-b7f7-8a033548b6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:17.200512 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c22338-ce85-418a-b7f7-8a033548b6f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:17.201303 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c22338-ce85-418a-b7f7-8a033548b6f2 None admin] RBAC: Adding query filter params (role.id=c4ba94fa4bd34a09ba87d357142d9db8, scope.project.id=400a983b4cd947d487b7fa86a5f5c339, user.id=b17ad37d9c5d445990fce41f2172124d) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:17.201618 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c22338-ce85-418a-b7f7-8a033548b6f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:17.203395 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c22338-ce85-418a-b7f7-8a033548b6f2 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:17.208990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 112/220] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:13:16 2022] GET /identity/v3/role_assignments?user.id=b17ad37d9c5d445990fce41f2172124d&scope.project.id=400a983b4cd947d487b7fa86a5f5c339&role.id=c4ba94fa4bd34a09ba87d357142d9db8 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:17.512144 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2013799c-56e3-499a-bd54-3bf3aa6f32ef None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:17.512679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2013799c-56e3-499a-bd54-3bf3aa6f32ef None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:17.512679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2013799c-56e3-499a-bd54-3bf3aa6f32ef None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:17.514051 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 109/221] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:17.519820 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4dc679-6846-4bd4-80ef-898540f6c87f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:17.520113 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4dc679-6846-4bd4-80ef-898540f6c87f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:17.520395 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4dc679-6846-4bd4-80ef-898540f6c87f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:17.539826 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ae4dc679-6846-4bd4-80ef-898540f6c87f None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:17.541012 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ae4dc679-6846-4bd4-80ef-898540f6c87f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:17.617737 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 113/222] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:17.639324 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2013799c-56e3-499a-bd54-3bf3aa6f32ef None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:17.729449 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:17.796924 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2013799c-56e3-499a-bd54-3bf3aa6f32ef None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:17.841415 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c1f24-02c1-4c17-8c9d-d12cf6c5f2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:17.842068 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c1f24-02c1-4c17-8c9d-d12cf6c5f2bd None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:17.842346 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c1f24-02c1-4c17-8c9d-d12cf6c5f2bd None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:17.842617 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c1f24-02c1-4c17-8c9d-d12cf6c5f2bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:17.859249 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5c1f24-02c1-4c17-8c9d-d12cf6c5f2bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:17.869042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-7e5c1f24-02c1-4c17-8c9d-d12cf6c5f2bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:13:17.869042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:17.869042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:17.869042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:17.869042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:17.869042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:17.869042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:17.869042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:17.869042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:17.869042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:17.869042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:17.869042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:17.869042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:17.869042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:17.869042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:17.869042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:17.869995 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:17.869995 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:17.869995 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:17.869995 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:17.869995 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:17.869995 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:17.869995 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:17.869995 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:17.869995 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:17.869995 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:17.869995 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:17.869995 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:17.869995 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:17.869995 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:17.869995 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:17.869995 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:17.871233 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:17.871233 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:17.871233 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:17.871233 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:17.871233 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:17.871233 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:17.871233 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:17.871233 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:17.871233 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:17.871233 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:17.871233 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:13:17.871233 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:17.871233 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 110/223] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 10:13:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:17.876424 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4dc679-6846-4bd4-80ef-898540f6c87f None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:17.971189 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3811f6da-1ff7-4882-943a-dd9a2fa2bf74 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:17.971535 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3811f6da-1ff7-4882-943a-dd9a2fa2bf74 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:17.971731 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3811f6da-1ff7-4882-943a-dd9a2fa2bf74 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:17.973346 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 111/224] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:17.986117 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:17.992074 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6c8d3-dc43-42a5-b7aa-951fb3c1f3cf None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:17.992354 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6c8d3-dc43-42a5-b7aa-951fb3c1f3cf None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:17.992632 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6c8d3-dc43-42a5-b7aa-951fb3c1f3cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:17.992831 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4dc679-6846-4bd4-80ef-898540f6c87f None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:18.027607 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a1241-b33b-4747-b88f-a786c949099d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:18.028237 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a1241-b33b-4747-b88f-a786c949099d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:18.028515 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a1241-b33b-4747-b88f-a786c949099d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:18.028787 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a1241-b33b-4747-b88f-a786c949099d None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:18.029509 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a1241-b33b-4747-b88f-a786c949099d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:18.029807 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a1241-b33b-4747-b88f-a786c949099d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:18.031370 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a1241-b33b-4747-b88f-a786c949099d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:18.032249 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-eab6c8d3-dc43-42a5-b7aa-951fb3c1f3cf None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:18.033657 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-eab6c8d3-dc43-42a5-b7aa-951fb3c1f3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:18.040452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 114/225] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:13:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:18.047135 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a1241-b33b-4747-b88f-a786c949099d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:18.164234 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 112/226] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:18.173632 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a1241-b33b-4747-b88f-a786c949099d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:18.186799 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6c8d3-dc43-42a5-b7aa-951fb3c1f3cf None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:18.221693 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d670c4-e752-468c-86a8-56d6e6f1b45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:18.224291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d670c4-e752-468c-86a8-56d6e6f1b45a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:18.224944 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d670c4-e752-468c-86a8-56d6e6f1b45a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:18.225608 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d670c4-e752-468c-86a8-56d6e6f1b45a None admin] PATH_INFO: `/v3/users/6e6904b0179c4a96bafa44a2c486016f` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:18.227749 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d670c4-e752-468c-86a8-56d6e6f1b45a None admin] RBAC: Authorizing `identity:get_user(user_id=6e6904b0179c4a96bafa44a2c486016f)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:18.230765 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d670c4-e752-468c-86a8-56d6e6f1b45a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:18.234773 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 115/227] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:13:18 2022] GET /identity/v3/users/6e6904b0179c4a96bafa44a2c486016f => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:18.241511 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d670c4-e752-468c-86a8-56d6e6f1b45a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:18.242458 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:18.295404 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:18.356969 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6c8d3-dc43-42a5-b7aa-951fb3c1f3cf None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:18.385815 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d670c4-e752-468c-86a8-56d6e6f1b45a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:18.426879 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11cedf4-bdcd-4aeb-9f26-5555c2fc8e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:18.428715 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11cedf4-bdcd-4aeb-9f26-5555c2fc8e2a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:18.429299 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11cedf4-bdcd-4aeb-9f26-5555c2fc8e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:18.430549 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11cedf4-bdcd-4aeb-9f26-5555c2fc8e2a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:18.441922 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11cedf4-bdcd-4aeb-9f26-5555c2fc8e2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:18.447365 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-d11cedf4-bdcd-4aeb-9f26-5555c2fc8e2a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 10:13:18.447365 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:18.447365 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:18.447365 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:18.447365 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:18.447365 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:18.447365 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:18.447365 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:18.447365 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:18.447365 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:18.447365 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:18.447365 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:18.447365 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:18.447365 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:18.447365 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:18.447365 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:18.448427 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:18.448427 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:18.448427 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:18.448427 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:18.448427 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:18.448427 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:18.448427 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:18.448427 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:18.448427 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:18.448427 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:18.448427 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:18.448427 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:18.448427 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:18.448427 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:18.448427 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:18.448427 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:18.449422 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:18.449422 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:18.449422 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:18.449422 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:18.449422 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:18.449422 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:18.449422 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:18.449422 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:18.449422 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:18.449422 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:18.449422 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 10:13:18.449422 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:18.451343 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 113/228] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 5 10:13:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:13:18.457583 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11cedf4-bdcd-4aeb-9f26-5555c2fc8e2a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:18.475609 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7951d418-3b41-4dfb-90bd-6bd6f37b7f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:18.476324 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951d418-3b41-4dfb-90bd-6bd6f37b7f80 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:18.476554 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951d418-3b41-4dfb-90bd-6bd6f37b7f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:18.476821 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951d418-3b41-4dfb-90bd-6bd6f37b7f80 None admin] PATH_INFO: `/v3/projects/400a983b4cd947d487b7fa86a5f5c339` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:18.478046 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7951d418-3b41-4dfb-90bd-6bd6f37b7f80 None admin] RBAC: Authorizing `identity:get_project(project_id=400a983b4cd947d487b7fa86a5f5c339)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:18.479489 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7951d418-3b41-4dfb-90bd-6bd6f37b7f80 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:18.481345 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 116/229] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:13:18 2022] GET /identity/v3/projects/400a983b4cd947d487b7fa86a5f5c339 => generated 302 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:18.491811 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7951d418-3b41-4dfb-90bd-6bd6f37b7f80 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:18.578490 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11cedf4-bdcd-4aeb-9f26-5555c2fc8e2a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:18.589542 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:18.616702 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d9873f-0576-489c-a436-51d105b140fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:18.617455 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d9873f-0576-489c-a436-51d105b140fd None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:18.617833 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d9873f-0576-489c-a436-51d105b140fd None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:18.618225 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d9873f-0576-489c-a436-51d105b140fd None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:18.619080 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d9873f-0576-489c-a436-51d105b140fd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:18.619584 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d9873f-0576-489c-a436-51d105b140fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:18.621080 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d9873f-0576-489c-a436-51d105b140fd None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:18.631648 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 114/230] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:13:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:13:18.637225 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d9873f-0576-489c-a436-51d105b140fd None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:18.745680 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d9873f-0576-489c-a436-51d105b140fd None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:18.783448 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c392910e-0426-4f04-b9a5-57c7a147c7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:18.784276 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c392910e-0426-4f04-b9a5-57c7a147c7c7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:18.784812 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c392910e-0426-4f04-b9a5-57c7a147c7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:18.785186 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c392910e-0426-4f04-b9a5-57c7a147c7c7 None admin] PATH_INFO: `/v3/users/6e6904b0179c4a96bafa44a2c486016f` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:18.786469 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c392910e-0426-4f04-b9a5-57c7a147c7c7 None admin] RBAC: Authorizing `identity:get_user(user_id=6e6904b0179c4a96bafa44a2c486016f)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:18.788502 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c392910e-0426-4f04-b9a5-57c7a147c7c7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:18.790709 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 115/231] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:13:18 2022] GET /identity/v3/users/6e6904b0179c4a96bafa44a2c486016f => generated 283 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:18.799873 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c392910e-0426-4f04-b9a5-57c7a147c7c7 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:18.811905 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:18.820927 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7951d418-3b41-4dfb-90bd-6bd6f37b7f80 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:18.873315 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:18.891406 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2937c7-9b9e-45cf-99f9-7ebece13aef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:18.892104 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2937c7-9b9e-45cf-99f9-7ebece13aef0 None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:18.892385 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2937c7-9b9e-45cf-99f9-7ebece13aef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:18.892660 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2937c7-9b9e-45cf-99f9-7ebece13aef0 None admin] PATH_INFO: `/v3/projects/400a983b4cd947d487b7fa86a5f5c339/users/6e6904b0179c4a96bafa44a2c486016f/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:18.895110 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2937c7-9b9e-45cf-99f9-7ebece13aef0 None admin] RBAC: Authorizing `identity:create_grant(project_id=400a983b4cd947d487b7fa86a5f5c339, user_id=6e6904b0179c4a96bafa44a2c486016f, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:18.896581 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2937c7-9b9e-45cf-99f9-7ebece13aef0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:18.923750 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 117/232] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 10:13:18 2022] PUT /identity/v3/projects/400a983b4cd947d487b7fa86a5f5c339/users/6e6904b0179c4a96bafa44a2c486016f/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:13:18.963889 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c392910e-0426-4f04-b9a5-57c7a147c7c7 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:18.999445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4e24e7-3abb-4c9b-9a2b-9378e1c0c02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:19.000112 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e24e7-3abb-4c9b-9a2b-9378e1c0c02d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:19.000396 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e24e7-3abb-4c9b-9a2b-9378e1c0c02d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:19.000667 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e24e7-3abb-4c9b-9a2b-9378e1c0c02d None admin] PATH_INFO: `/v3/projects/400a983b4cd947d487b7fa86a5f5c339` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:19.001867 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4e24e7-3abb-4c9b-9a2b-9378e1c0c02d None admin] RBAC: Authorizing `identity:get_project(project_id=400a983b4cd947d487b7fa86a5f5c339)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:19.003372 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4e24e7-3abb-4c9b-9a2b-9378e1c0c02d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:19.005058 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 116/233] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:13:18 2022] GET /identity/v3/projects/400a983b4cd947d487b7fa86a5f5c339 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:19.012251 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2937c7-9b9e-45cf-99f9-7ebece13aef0 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:19.150480 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2eff34-c9c0-4a53-ba27-c264c3bb6901 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:19.150929 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2eff34-c9c0-4a53-ba27-c264c3bb6901 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:19.151299 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2eff34-c9c0-4a53-ba27-c264c3bb6901 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:19.155025 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 117/234] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:19.156980 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2937c7-9b9e-45cf-99f9-7ebece13aef0 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:19.167423 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde32196-3d3b-4f03-9df3-7b4131a6bbd7 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:19.167697 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde32196-3d3b-4f03-9df3-7b4131a6bbd7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:19.167962 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde32196-3d3b-4f03-9df3-7b4131a6bbd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:19.190332 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-fde32196-3d3b-4f03-9df3-7b4131a6bbd7 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:19.191542 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fde32196-3d3b-4f03-9df3-7b4131a6bbd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:19.195633 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e093931-c040-467d-8986-9941ef66b599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:19.196322 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e093931-c040-467d-8986-9941ef66b599 None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:19.196606 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e093931-c040-467d-8986-9941ef66b599 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:19.196881 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e093931-c040-467d-8986-9941ef66b599 None admin] PATH_INFO: `/v3/projects/400a983b4cd947d487b7fa86a5f5c339/users/6e6904b0179c4a96bafa44a2c486016f/roles/57a070b43ad940f1886c69721dd304c2` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:19.200795 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e093931-c040-467d-8986-9941ef66b599 None admin] RBAC: Authorizing `identity:create_grant(project_id=400a983b4cd947d487b7fa86a5f5c339, user_id=6e6904b0179c4a96bafa44a2c486016f, role_id=57a070b43ad940f1886c69721dd304c2)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:19.202641 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e093931-c040-467d-8986-9941ef66b599 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:19.229481 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 118/235] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 10:13:19 2022] PUT /identity/v3/projects/400a983b4cd947d487b7fa86a5f5c339/users/6e6904b0179c4a96bafa44a2c486016f/roles/57a070b43ad940f1886c69721dd304c2 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:13:19.282258 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 118/236] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:19.306265 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e093931-c040-467d-8986-9941ef66b599 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:19.323692 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:19.347257 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e790b33-9c36-4dda-85be-64b025069cf7 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:19.347875 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e790b33-9c36-4dda-85be-64b025069cf7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:19.348331 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e790b33-9c36-4dda-85be-64b025069cf7 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:19.349934 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 119/237] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:19.356276 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14084f98-a51c-487d-a589-37f85ef75f84 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:19.356641 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14084f98-a51c-487d-a589-37f85ef75f84 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:19.356925 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14084f98-a51c-487d-a589-37f85ef75f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:19.400112 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-14084f98-a51c-487d-a589-37f85ef75f84 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:19.401384 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-14084f98-a51c-487d-a589-37f85ef75f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:19.434452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e093931-c040-467d-8986-9941ef66b599 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:19.459107 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:19.476101 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea7112-95e4-49b2-b164-70964053a0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:19.477143 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea7112-95e4-49b2-b164-70964053a0e4 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:19.477612 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea7112-95e4-49b2-b164-70964053a0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:19.478070 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea7112-95e4-49b2-b164-70964053a0e4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:19.491245 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daea7112-95e4-49b2-b164-70964053a0e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:19.498244 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-daea7112-95e4-49b2-b164-70964053a0e4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 10:13:19.498244 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:19.498244 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:19.498244 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:19.498244 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:19.498244 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:19.498244 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:19.498244 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:19.498244 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:19.498244 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:19.498244 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:19.498244 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:19.498244 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:19.498244 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:19.498244 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:19.498244 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:19.499270 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:19.499270 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:19.499270 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:19.499270 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:19.499270 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:19.499270 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:19.499270 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:19.499270 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:19.499270 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:19.499270 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:19.499270 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:19.499270 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:19.499270 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:19.499270 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:19.499270 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:19.499270 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:19.500157 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:19.500157 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:19.500157 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:19.500157 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:19.500157 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:19.500157 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:19.500157 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:19.500157 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:19.500157 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:19.500157 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:19.500157 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 10:13:19.500157 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:13:19.502843 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 119/238] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:13:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:19.509355 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea7112-95e4-49b2-b164-70964053a0e4 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:19.509829 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 120/239] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:19.535890 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14084f98-a51c-487d-a589-37f85ef75f84 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:19.583500 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:19.614521 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea7112-95e4-49b2-b164-70964053a0e4 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:19.649753 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14084f98-a51c-487d-a589-37f85ef75f84 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:19.655163 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db588124-3264-4925-a4b4-90316f18fbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:19.656344 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db588124-3264-4925-a4b4-90316f18fbb3 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:19.656821 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db588124-3264-4925-a4b4-90316f18fbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:19.657326 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db588124-3264-4925-a4b4-90316f18fbb3 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:19.658338 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db588124-3264-4925-a4b4-90316f18fbb3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:19.658830 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db588124-3264-4925-a4b4-90316f18fbb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:19.660536 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db588124-3264-4925-a4b4-90316f18fbb3 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:19.670554 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 120/240] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 10:13:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:13:19.677353 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db588124-3264-4925-a4b4-90316f18fbb3 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:19.697580 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e7a212-1990-48f0-b354-e2b266bff6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:19.698626 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e7a212-1990-48f0-b354-e2b266bff6d0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:19.699444 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e7a212-1990-48f0-b354-e2b266bff6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:19.700096 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e7a212-1990-48f0-b354-e2b266bff6d0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:19.703772 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e7a212-1990-48f0-b354-e2b266bff6d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:19.706512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e7a212-1990-48f0-b354-e2b266bff6d0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:19.711654 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 121/241] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:13:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:13:19.715583 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e7a212-1990-48f0-b354-e2b266bff6d0 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:19.777310 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:19.812273 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db588124-3264-4925-a4b4-90316f18fbb3 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:19.852903 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe83287-f8af-4a7f-87d7-7fd0d17f79cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:19.853566 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe83287-f8af-4a7f-87d7-7fd0d17f79cc None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:19.853849 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe83287-f8af-4a7f-87d7-7fd0d17f79cc None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:19.854113 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe83287-f8af-4a7f-87d7-7fd0d17f79cc None admin] PATH_INFO: `/v3/users/e2051851b02d431891a5a0ccd79b860f` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:19.861052 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:19.875506 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe83287-f8af-4a7f-87d7-7fd0d17f79cc None admin] RBAC: Authorizing `identity:get_user(user_id=e2051851b02d431891a5a0ccd79b860f)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:19.877272 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe83287-f8af-4a7f-87d7-7fd0d17f79cc None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:19.880700 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 121/242] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:13:19 2022] GET /identity/v3/users/e2051851b02d431891a5a0ccd79b860f => generated 297 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:19.886361 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe83287-f8af-4a7f-87d7-7fd0d17f79cc None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:19.922786 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e7a212-1990-48f0-b354-e2b266bff6d0 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:19.960798 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5668cecc-8025-4064-a360-6ec73c3d1fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:19.961572 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5668cecc-8025-4064-a360-6ec73c3d1fef None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:19.961923 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5668cecc-8025-4064-a360-6ec73c3d1fef None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:19.962201 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5668cecc-8025-4064-a360-6ec73c3d1fef None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:19.962910 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5668cecc-8025-4064-a360-6ec73c3d1fef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:19.964678 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5668cecc-8025-4064-a360-6ec73c3d1fef None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:19.995295 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 122/243] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 5 10:13:19 2022] POST /identity/v3/users => generated 293 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:13:19.996725 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf0a9b9-5a8f-447a-8148-0b1e225c0971 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:19.997072 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf0a9b9-5a8f-447a-8148-0b1e225c0971 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:19.997340 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf0a9b9-5a8f-447a-8148-0b1e225c0971 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:19.998959 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 123/244] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:20.004519 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe83287-f8af-4a7f-87d7-7fd0d17f79cc None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:20.008725 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c71fdd-a2d9-4124-b94f-e670f7bb8907 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:20.009012 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c71fdd-a2d9-4124-b94f-e670f7bb8907 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:20.009293 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c71fdd-a2d9-4124-b94f-e670f7bb8907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:20.032499 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-07c71fdd-a2d9-4124-b94f-e670f7bb8907 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:20.033775 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-07c71fdd-a2d9-4124-b94f-e670f7bb8907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:20.065055 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7216f8fd-4044-427a-b1cc-b7a464b986d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:20.065699 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216f8fd-4044-427a-b1cc-b7a464b986d7 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:20.065980 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216f8fd-4044-427a-b1cc-b7a464b986d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:20.066243 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216f8fd-4044-427a-b1cc-b7a464b986d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:20.110076 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 124/245] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:20.131866 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c71fdd-a2d9-4124-b94f-e670f7bb8907 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:20.151482 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:20.189847 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:20.197284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7216f8fd-4044-427a-b1cc-b7a464b986d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:20.198780 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7216f8fd-4044-427a-b1cc-b7a464b986d7 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:20.255292 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c71fdd-a2d9-4124-b94f-e670f7bb8907 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:20.297735 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb50f4-4c01-4bab-aabd-c5e866d3f3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:20.298782 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb50f4-4c01-4bab-aabd-c5e866d3f3aa None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:20.301695 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb50f4-4c01-4bab-aabd-c5e866d3f3aa None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:20.302175 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb50f4-4c01-4bab-aabd-c5e866d3f3aa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:20.321935 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeb50f4-4c01-4bab-aabd-c5e866d3f3aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:20.329116 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-4aeb50f4-4c01-4bab-aabd-c5e866d3f3aa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 10:13:20.329116 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:20.329116 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:20.329116 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:20.329116 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:20.329116 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:20.329116 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:20.329116 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:20.329116 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:20.329116 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:20.329116 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:20.329116 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:20.329116 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:20.329116 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:20.329116 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:20.329116 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:20.330014 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:20.330014 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:20.330014 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:20.330014 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:20.330014 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:20.330014 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:20.330014 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:20.330014 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:20.330014 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:20.330014 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:20.330014 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:20.330014 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:20.330014 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:20.330014 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:20.330014 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:20.330014 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:20.330884 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:20.330884 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:20.330884 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:20.330884 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:20.330884 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:20.330884 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:20.330884 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:20.330884 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:20.330884 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:20.330884 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:20.330884 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 10:13:20.330884 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:20.330884 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 125/246] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:13:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:13:20.337911 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb50f4-4c01-4bab-aabd-c5e866d3f3aa None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:20.379534 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 122/247] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:13:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 10:13:20.385673 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7216f8fd-4044-427a-b1cc-b7a464b986d7 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:20.439336 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:20.460490 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb50f4-4c01-4bab-aabd-c5e866d3f3aa None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:20.491866 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:20.501671 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7216f8fd-4044-427a-b1cc-b7a464b986d7 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:20.508064 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb61ffc-16bb-4d41-b944-083fce5fd3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:20.508982 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb61ffc-16bb-4d41-b944-083fce5fd3c8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:20.509458 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb61ffc-16bb-4d41-b944-083fce5fd3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:20.509924 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb61ffc-16bb-4d41-b944-083fce5fd3c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:20.511384 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb61ffc-16bb-4d41-b944-083fce5fd3c8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:20.511895 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb61ffc-16bb-4d41-b944-083fce5fd3c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:20.513976 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb61ffc-16bb-4d41-b944-083fce5fd3c8 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:20.525539 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 126/248] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 10:13:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:13:20.531557 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb61ffc-16bb-4d41-b944-083fce5fd3c8 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:20.543091 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a576d2a5-cbc4-4604-8834-a3f55bd62a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:20.543745 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576d2a5-cbc4-4604-8834-a3f55bd62a0b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:20.544019 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576d2a5-cbc4-4604-8834-a3f55bd62a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:20.544291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576d2a5-cbc4-4604-8834-a3f55bd62a0b None admin] PATH_INFO: `/v3/projects/400a983b4cd947d487b7fa86a5f5c339` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:20.545798 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a576d2a5-cbc4-4604-8834-a3f55bd62a0b None admin] RBAC: Authorizing `identity:get_project(project_id=400a983b4cd947d487b7fa86a5f5c339)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:20.547343 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a576d2a5-cbc4-4604-8834-a3f55bd62a0b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:20.549046 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 123/249] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:13:20 2022] GET /identity/v3/projects/400a983b4cd947d487b7fa86a5f5c339 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:20.555187 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a576d2a5-cbc4-4604-8834-a3f55bd62a0b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:20.642677 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb61ffc-16bb-4d41-b944-083fce5fd3c8 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:20.663140 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a576d2a5-cbc4-4604-8834-a3f55bd62a0b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:20.682528 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61469125-14b8-4434-95d7-3ddd4f115c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:20.683919 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61469125-14b8-4434-95d7-3ddd4f115c5f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:20.684458 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61469125-14b8-4434-95d7-3ddd4f115c5f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:20.684922 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61469125-14b8-4434-95d7-3ddd4f115c5f None admin] PATH_INFO: `/v3/users/b17ad37d9c5d445990fce41f2172124d` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:20.686333 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61469125-14b8-4434-95d7-3ddd4f115c5f None admin] RBAC: Authorizing `identity:get_user(user_id=b17ad37d9c5d445990fce41f2172124d)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:20.688477 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61469125-14b8-4434-95d7-3ddd4f115c5f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:20.690641 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 127/250] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:13:20 2022] GET /identity/v3/users/b17ad37d9c5d445990fce41f2172124d => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:13:20.696353 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61469125-14b8-4434-95d7-3ddd4f115c5f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:20.699705 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6e0b4a-6499-4b97-8ede-f4e8f4cd2061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:20.700300 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e0b4a-6499-4b97-8ede-f4e8f4cd2061 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:20.700577 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e0b4a-6499-4b97-8ede-f4e8f4cd2061 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:20.700844 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e0b4a-6499-4b97-8ede-f4e8f4cd2061 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:20.702137 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6e0b4a-6499-4b97-8ede-f4e8f4cd2061 None admin] RBAC: Adding query filter params (role.id=cf5de414a035475b8b75626cb19a2de2, scope.project.id=400a983b4cd947d487b7fa86a5f5c339, user.id=e2051851b02d431891a5a0ccd79b860f) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:20.702137 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6e0b4a-6499-4b97-8ede-f4e8f4cd2061 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:20.704741 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6e0b4a-6499-4b97-8ede-f4e8f4cd2061 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:20.716325 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 124/251] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:13:20 2022] GET /identity/v3/role_assignments?user.id=e2051851b02d431891a5a0ccd79b860f&scope.project.id=400a983b4cd947d487b7fa86a5f5c339&role.id=cf5de414a035475b8b75626cb19a2de2 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:13:20.721497 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:20.816527 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61469125-14b8-4434-95d7-3ddd4f115c5f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:20.857859 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51aa3f0-b034-40d3-b8d2-1a1daff65f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:20.859008 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51aa3f0-b034-40d3-b8d2-1a1daff65f78 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:20.859585 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51aa3f0-b034-40d3-b8d2-1a1daff65f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:20.860163 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51aa3f0-b034-40d3-b8d2-1a1daff65f78 None admin] PATH_INFO: `/v3/projects/400a983b4cd947d487b7fa86a5f5c339` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:20.861670 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51aa3f0-b034-40d3-b8d2-1a1daff65f78 None admin] RBAC: Authorizing `identity:get_project(project_id=400a983b4cd947d487b7fa86a5f5c339)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:20.863408 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51aa3f0-b034-40d3-b8d2-1a1daff65f78 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:20.865733 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 128/252] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:13:20 2022] GET /identity/v3/projects/400a983b4cd947d487b7fa86a5f5c339 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:20.871434 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6e0b4a-6499-4b97-8ede-f4e8f4cd2061 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:20.902280 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:21.005215 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6e0b4a-6499-4b97-8ede-f4e8f4cd2061 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:21.059420 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c289f4-b290-4a4e-848b-7df14cdfab11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:21.060118 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c289f4-b290-4a4e-848b-7df14cdfab11 None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:21.060402 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c289f4-b290-4a4e-848b-7df14cdfab11 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:21.060675 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c289f4-b290-4a4e-848b-7df14cdfab11 None admin] PATH_INFO: `/v3/projects/400a983b4cd947d487b7fa86a5f5c339/users/b17ad37d9c5d445990fce41f2172124d/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:21.064748 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c289f4-b290-4a4e-848b-7df14cdfab11 None admin] RBAC: Authorizing `identity:create_grant(project_id=400a983b4cd947d487b7fa86a5f5c339, user_id=b17ad37d9c5d445990fce41f2172124d, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:21.066220 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c289f4-b290-4a4e-848b-7df14cdfab11 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:21.100100 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 125/253] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 10:13:20 2022] PUT /identity/v3/projects/400a983b4cd947d487b7fa86a5f5c339/users/b17ad37d9c5d445990fce41f2172124d/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:13:21.380434 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6017c6-ba24-48a2-a70a-5cc062293dff None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:21.381107 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6017c6-ba24-48a2-a70a-5cc062293dff None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:21.381582 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6017c6-ba24-48a2-a70a-5cc062293dff None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:21.383200 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 129/254] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:21.388948 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ed442-ab2a-4d92-87e9-d0a5fe68f71a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:21.389520 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ed442-ab2a-4d92-87e9-d0a5fe68f71a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:21.390010 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ed442-ab2a-4d92-87e9-d0a5fe68f71a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:21.409140 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-d74ed442-ab2a-4d92-87e9-d0a5fe68f71a None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:21.410677 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d74ed442-ab2a-4d92-87e9-d0a5fe68f71a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:21.492268 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 126/255] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:21.530620 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6017c6-ba24-48a2-a70a-5cc062293dff None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:21.632098 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:21.666645 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6017c6-ba24-48a2-a70a-5cc062293dff None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:21.702588 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4512bcf3-4fb0-4b57-b03a-ab83f604b70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:21.704190 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512bcf3-4fb0-4b57-b03a-ab83f604b70e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:21.704658 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512bcf3-4fb0-4b57-b03a-ab83f604b70e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:21.705112 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512bcf3-4fb0-4b57-b03a-ab83f604b70e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:21.716155 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4512bcf3-4fb0-4b57-b03a-ab83f604b70e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:21.722005 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-4512bcf3-4fb0-4b57-b03a-ab83f604b70e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:13:21.722005 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:21.722005 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:21.722005 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:21.722005 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:21.722005 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:21.722005 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:21.722005 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:21.722005 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:21.722005 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:21.722005 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:21.722005 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:21.722005 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:21.722005 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:21.722005 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:21.722005 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:21.722863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:21.722863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:21.722863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:21.722863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:21.722863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:21.722863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:21.722863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:21.722863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:21.722863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:21.722863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:21.722863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:21.722863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:21.722863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:21.722863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:21.722863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:21.722863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:21.723677 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:21.723677 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:21.723677 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:21.723677 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:21.723677 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:21.723677 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:21.723677 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:21.723677 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:21.723677 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:21.723677 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:21.723677 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:13:21.723677 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:21.726299 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 130/256] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 10:13:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:21.732548 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74ed442-ab2a-4d92-87e9-d0a5fe68f71a None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:21.752293 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:21.783414 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d544da-8c1d-4118-a957-abcd23fec48a None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:21.784041 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d544da-8c1d-4118-a957-abcd23fec48a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:21.784520 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d544da-8c1d-4118-a957-abcd23fec48a None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:21.786065 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 131/257] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 10:13:21.791262 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b9ec4c-e882-43ec-bc13-b4cec94ff71f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:21.791795 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b9ec4c-e882-43ec-bc13-b4cec94ff71f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:21.792255 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b9ec4c-e882-43ec-bc13-b4cec94ff71f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:21.817562 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-36b9ec4c-e882-43ec-bc13-b4cec94ff71f None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:21.821776 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-36b9ec4c-e882-43ec-bc13-b4cec94ff71f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:21.897671 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74ed442-ab2a-4d92-87e9-d0a5fe68f71a None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:21.975650 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db035955-7077-449d-8e4d-f2a3d4ea9214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:21.976299 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db035955-7077-449d-8e4d-f2a3d4ea9214 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:21.976580 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db035955-7077-449d-8e4d-f2a3d4ea9214 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:21.976851 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db035955-7077-449d-8e4d-f2a3d4ea9214 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:21.977586 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db035955-7077-449d-8e4d-f2a3d4ea9214 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:21.977889 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db035955-7077-449d-8e4d-f2a3d4ea9214 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:21.979315 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db035955-7077-449d-8e4d-f2a3d4ea9214 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:21.994007 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 127/258] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:13:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:22.000474 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db035955-7077-449d-8e4d-f2a3d4ea9214 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:22.009173 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 132/259] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 10:13:22.044554 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b9ec4c-e882-43ec-bc13-b4cec94ff71f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:22.143701 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:22.163390 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:22.172972 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db035955-7077-449d-8e4d-f2a3d4ea9214 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:22.233029 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b9ec4c-e882-43ec-bc13-b4cec94ff71f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:22.236160 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615cec8-874c-4bd4-a08f-148a93d7f12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:22.236844 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615cec8-874c-4bd4-a08f-148a93d7f12d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:22.237178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615cec8-874c-4bd4-a08f-148a93d7f12d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:22.237461 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615cec8-874c-4bd4-a08f-148a93d7f12d None admin] PATH_INFO: `/v3/users/6e6904b0179c4a96bafa44a2c486016f` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:22.238659 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d615cec8-874c-4bd4-a08f-148a93d7f12d None admin] RBAC: Authorizing `identity:get_user(user_id=6e6904b0179c4a96bafa44a2c486016f)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:22.240237 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d615cec8-874c-4bd4-a08f-148a93d7f12d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:22.241858 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 128/260] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:13:21 2022] GET /identity/v3/users/6e6904b0179c4a96bafa44a2c486016f => generated 283 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:13:22.250771 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615cec8-874c-4bd4-a08f-148a93d7f12d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:22.319933 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f898c7-5a5e-49fa-a740-6add470e05bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:22.320653 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f898c7-5a5e-49fa-a740-6add470e05bc None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:22.320934 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f898c7-5a5e-49fa-a740-6add470e05bc None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:22.321206 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f898c7-5a5e-49fa-a740-6add470e05bc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:22.360489 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f898c7-5a5e-49fa-a740-6add470e05bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:22.365530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-06f898c7-5a5e-49fa-a740-6add470e05bc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 10:13:22.365530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:22.365530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:22.365530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:22.365530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:22.365530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:22.365530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:22.365530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:22.365530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:22.365530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:22.365530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:22.365530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:22.365530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:22.365530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:22.365530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:22.365530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:22.366485 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:22.366485 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:22.366485 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:22.366485 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:22.366485 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:22.366485 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:22.366485 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:22.366485 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:22.366485 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:22.366485 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:22.366485 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:22.366485 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:22.366485 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:22.366485 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:22.366485 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:22.366485 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:22.367293 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:22.367293 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:22.367293 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:22.367293 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:22.367293 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:22.367293 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:22.367293 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:22.367293 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:22.367293 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:22.367293 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:22.367293 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 10:13:22.367293 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:22.367293 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 133/261] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 5 10:13:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:22.375300 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f898c7-5a5e-49fa-a740-6add470e05bc None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:22.396115 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615cec8-874c-4bd4-a08f-148a93d7f12d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:22.450669 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a8641-f9ea-4b43-901a-d1a10a5c4da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:22.451266 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a8641-f9ea-4b43-901a-d1a10a5c4da0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:22.451548 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a8641-f9ea-4b43-901a-d1a10a5c4da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:22.451824 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a8641-f9ea-4b43-901a-d1a10a5c4da0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:22.477607 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:22.564376 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489a8641-f9ea-4b43-901a-d1a10a5c4da0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:22.565920 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489a8641-f9ea-4b43-901a-d1a10a5c4da0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:22.634564 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f898c7-5a5e-49fa-a740-6add470e05bc None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:22.700471 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:22.728514 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9169fb04-1694-46b0-8da0-af7e2fb90203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:22.729904 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9169fb04-1694-46b0-8da0-af7e2fb90203 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:22.730381 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9169fb04-1694-46b0-8da0-af7e2fb90203 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:22.730872 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9169fb04-1694-46b0-8da0-af7e2fb90203 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:22.731823 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9169fb04-1694-46b0-8da0-af7e2fb90203 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:22.732347 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9169fb04-1694-46b0-8da0-af7e2fb90203 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:22.734010 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9169fb04-1694-46b0-8da0-af7e2fb90203 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:22.752665 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 134/262] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:13:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:22.757087 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9169fb04-1694-46b0-8da0-af7e2fb90203 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:22.783752 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 129/263] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:13:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 10:13:22.789875 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a8641-f9ea-4b43-901a-d1a10a5c4da0 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:22.843240 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:22.910947 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9169fb04-1694-46b0-8da0-af7e2fb90203 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:22.947780 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f7da13-005b-4554-ae4e-4ca273200232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:22.948436 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f7da13-005b-4554-ae4e-4ca273200232 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:22.948715 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f7da13-005b-4554-ae4e-4ca273200232 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:22.948984 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f7da13-005b-4554-ae4e-4ca273200232 None admin] PATH_INFO: `/v3/users/6e6904b0179c4a96bafa44a2c486016f` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:22.950171 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f7da13-005b-4554-ae4e-4ca273200232 None admin] RBAC: Authorizing `identity:get_user(user_id=6e6904b0179c4a96bafa44a2c486016f)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:22.951663 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f7da13-005b-4554-ae4e-4ca273200232 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:22.953683 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 135/264] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:13:22 2022] GET /identity/v3/users/6e6904b0179c4a96bafa44a2c486016f => generated 283 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:22.959125 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f7da13-005b-4554-ae4e-4ca273200232 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:22.961843 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a8641-f9ea-4b43-901a-d1a10a5c4da0 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:23.017627 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:23.059958 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b54635-7fa2-4429-82ed-d2f46f653c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:23.061117 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b54635-7fa2-4429-82ed-d2f46f653c50 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:23.061909 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b54635-7fa2-4429-82ed-d2f46f653c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:23.062384 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b54635-7fa2-4429-82ed-d2f46f653c50 None admin] PATH_INFO: `/v3/projects/400a983b4cd947d487b7fa86a5f5c339` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:23.063921 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b54635-7fa2-4429-82ed-d2f46f653c50 None admin] RBAC: Authorizing `identity:get_project(project_id=400a983b4cd947d487b7fa86a5f5c339)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:23.065228 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f7da13-005b-4554-ae4e-4ca273200232 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:23.065891 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b54635-7fa2-4429-82ed-d2f46f653c50 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:23.068610 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 130/265] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:13:22 2022] GET /identity/v3/projects/400a983b4cd947d487b7fa86a5f5c339 => generated 302 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:13:23.089352 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b54635-7fa2-4429-82ed-d2f46f653c50 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:23.108957 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44427511-7a70-4de2-84cb-568d5aa355f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:23.110313 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44427511-7a70-4de2-84cb-568d5aa355f3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:23.110597 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44427511-7a70-4de2-84cb-568d5aa355f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:23.110870 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44427511-7a70-4de2-84cb-568d5aa355f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:23.204535 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44427511-7a70-4de2-84cb-568d5aa355f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:23.206276 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44427511-7a70-4de2-84cb-568d5aa355f3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:23.257748 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b54635-7fa2-4429-82ed-d2f46f653c50 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:23.264033 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:23.275395 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:23.310365 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fba150f-81f5-47df-b0d9-7758a441794d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:23.311053 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba150f-81f5-47df-b0d9-7758a441794d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:23.311313 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba150f-81f5-47df-b0d9-7758a441794d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:23.311639 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba150f-81f5-47df-b0d9-7758a441794d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:23.312567 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fba150f-81f5-47df-b0d9-7758a441794d None admin] RBAC: Adding query filter params (role.id=daef89adcaeb48a382a0bcb03340ccbb, scope.project.id=400a983b4cd947d487b7fa86a5f5c339, user.id=6e6904b0179c4a96bafa44a2c486016f) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:23.312879 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fba150f-81f5-47df-b0d9-7758a441794d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:23.314352 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fba150f-81f5-47df-b0d9-7758a441794d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:23.314709 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 136/266] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:13:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 10:13:23.320477 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44427511-7a70-4de2-84cb-568d5aa355f3 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:23.326364 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 131/267] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:13:23 2022] GET /identity/v3/role_assignments?user.id=6e6904b0179c4a96bafa44a2c486016f&scope.project.id=400a983b4cd947d487b7fa86a5f5c339&role.id=daef89adcaeb48a382a0bcb03340ccbb => generated 608 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:23.445645 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44427511-7a70-4de2-84cb-568d5aa355f3 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:23.488240 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5988356-6bea-444b-873e-70bda0ae38a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:23.489230 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5988356-6bea-444b-873e-70bda0ae38a4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:23.489702 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5988356-6bea-444b-873e-70bda0ae38a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:23.490109 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5988356-6bea-444b-873e-70bda0ae38a4 None admin] PATH_INFO: `/v3/projects/400a983b4cd947d487b7fa86a5f5c339` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:23.491475 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5988356-6bea-444b-873e-70bda0ae38a4 None admin] RBAC: Authorizing `identity:get_project(project_id=400a983b4cd947d487b7fa86a5f5c339)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:23.493110 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5988356-6bea-444b-873e-70bda0ae38a4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:23.495444 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 137/268] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:13:23 2022] GET /identity/v3/projects/400a983b4cd947d487b7fa86a5f5c339 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:23.501903 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fba150f-81f5-47df-b0d9-7758a441794d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:23.628860 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fba150f-81f5-47df-b0d9-7758a441794d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:23.679738 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c32c40c-5ca1-48bc-a7f0-833798ce5f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:23.680376 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c32c40c-5ca1-48bc-a7f0-833798ce5f8a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:23.680660 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c32c40c-5ca1-48bc-a7f0-833798ce5f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:23.680937 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c32c40c-5ca1-48bc-a7f0-833798ce5f8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:23.681733 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c32c40c-5ca1-48bc-a7f0-833798ce5f8a None admin] RBAC: Adding query filter params (role.id=57a070b43ad940f1886c69721dd304c2, scope.project.id=400a983b4cd947d487b7fa86a5f5c339, user.id=6e6904b0179c4a96bafa44a2c486016f) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:23.682042 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c32c40c-5ca1-48bc-a7f0-833798ce5f8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:23.683542 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c32c40c-5ca1-48bc-a7f0-833798ce5f8a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:23.693452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 132/269] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:13:23 2022] GET /identity/v3/role_assignments?user.id=6e6904b0179c4a96bafa44a2c486016f&scope.project.id=400a983b4cd947d487b7fa86a5f5c339&role.id=57a070b43ad940f1886c69721dd304c2 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:23.737173 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb56ed-1de4-4322-9a06-bc3206a9aae4 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:23.737699 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb56ed-1de4-4322-9a06-bc3206a9aae4 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:23.738228 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb56ed-1de4-4322-9a06-bc3206a9aae4 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:23.740529 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 138/270] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:23.750292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b275c3-ffdd-404a-8e2c-0accb0032bc4 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:23.750644 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b275c3-ffdd-404a-8e2c-0accb0032bc4 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:23.750854 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b275c3-ffdd-404a-8e2c-0accb0032bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:23.772541 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c6b275c3-ffdd-404a-8e2c-0accb0032bc4 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:23.773760 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c6b275c3-ffdd-404a-8e2c-0accb0032bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:23.816651 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:23.863851 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 133/271] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:23.905647 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb56ed-1de4-4322-9a06-bc3206a9aae4 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:23.927824 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438cff1b-9566-4f71-989e-83525f6c212e None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:23.928148 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438cff1b-9566-4f71-989e-83525f6c212e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:23.928378 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438cff1b-9566-4f71-989e-83525f6c212e None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:23.930038 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 134/272] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:23.937907 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a3f0c-3931-44e8-90e3-91c346ecff80 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:23.938246 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a3f0c-3931-44e8-90e3-91c346ecff80 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:23.938461 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a3f0c-3931-44e8-90e3-91c346ecff80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:23.950758 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:23.974551 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-3b9a3f0c-3931-44e8-90e3-91c346ecff80 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:23.975798 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3b9a3f0c-3931-44e8-90e3-91c346ecff80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:24.072902 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb56ed-1de4-4322-9a06-bc3206a9aae4 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:24.078026 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 135/273] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:24.084492 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccf6ab-659b-4a6e-a29a-b58169aad3a5 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:24.085011 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccf6ab-659b-4a6e-a29a-b58169aad3a5 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:24.085498 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccf6ab-659b-4a6e-a29a-b58169aad3a5 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:24.087189 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 136/274] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:24.094160 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1830f-e024-43fe-9e41-d6bd9cbf10f6 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:24.094445 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1830f-e024-43fe-9e41-d6bd9cbf10f6 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:24.094823 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1830f-e024-43fe-9e41-d6bd9cbf10f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:24.116947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-0db1830f-e024-43fe-9e41-d6bd9cbf10f6 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:24.118117 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0db1830f-e024-43fe-9e41-d6bd9cbf10f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:24.120669 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d807e6-4c44-4afd-88c0-153c91e409e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:24.122154 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d807e6-4c44-4afd-88c0-153c91e409e7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:24.122441 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d807e6-4c44-4afd-88c0-153c91e409e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:24.122717 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d807e6-4c44-4afd-88c0-153c91e409e7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:24.136451 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d807e6-4c44-4afd-88c0-153c91e409e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:24.142853 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-e5d807e6-4c44-4afd-88c0-153c91e409e7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 10:13:24.142853 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:24.142853 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:24.142853 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:24.142853 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:24.142853 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:24.142853 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:24.142853 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:24.142853 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:24.142853 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:24.142853 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:24.142853 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:24.142853 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:24.142853 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:24.142853 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:24.142853 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:24.143847 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:24.143847 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:24.143847 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:24.143847 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:24.143847 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:24.143847 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:24.143847 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:24.143847 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:24.143847 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:24.143847 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:24.143847 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:24.143847 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:24.143847 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:24.143847 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:24.143847 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:24.143847 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:24.144635 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:24.144635 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:24.144635 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:24.144635 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:24.144635 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:24.144635 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:24.144635 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:24.144635 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:24.144635 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:24.144635 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:24.144635 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 10:13:24.144635 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:24.145436 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 139/275] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:13:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:24.146037 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d807e6-4c44-4afd-88c0-153c91e409e7 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:24.194947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 137/276] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 10:13:24.194947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db1830f-e024-43fe-9e41-d6bd9cbf10f6 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:24.224829 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:24.247373 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:24.264454 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d807e6-4c44-4afd-88c0-153c91e409e7 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:24.294092 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db1830f-e024-43fe-9e41-d6bd9cbf10f6 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:24.299807 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5840361f-ac81-493e-875b-13467c362638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:24.300455 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840361f-ac81-493e-875b-13467c362638 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:24.300735 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840361f-ac81-493e-875b-13467c362638 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:24.301005 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840361f-ac81-493e-875b-13467c362638 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:24.311286 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5840361f-ac81-493e-875b-13467c362638 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:24.316701 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-5840361f-ac81-493e-875b-13467c362638 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 10:13:24.316701 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:24.316701 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:24.316701 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:24.316701 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:24.316701 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:24.316701 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:24.316701 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:24.316701 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:24.316701 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:24.316701 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:24.316701 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:24.316701 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:24.316701 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:24.316701 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:24.316701 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:24.317796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:24.317796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:24.317796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:24.317796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:24.317796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:24.317796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:24.317796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:24.317796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:24.317796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:24.317796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:24.317796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:24.317796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:24.317796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:24.317796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:24.317796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:24.317796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:24.318577 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:24.318577 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:24.318577 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:24.318577 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:24.318577 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:24.318577 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:24.318577 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:24.318577 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:24.318577 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:24.318577 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:24.318577 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 10:13:24.318577 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:24.318577 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 140/277] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:13:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:24.319278 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279901d-f1c9-43e6-90e3-798e92a7153b None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:24.319346 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279901d-f1c9-43e6-90e3-798e92a7153b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:24.319635 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279901d-f1c9-43e6-90e3-798e92a7153b None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:24.320839 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 141/278] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:24.322808 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b279901d-f1c9-43e6-90e3-798e92a7153b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:24.334085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e9710-4afc-42c9-a9ad-8857339261c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:24.334713 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e9710-4afc-42c9-a9ad-8857339261c9 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:24.335037 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e9710-4afc-42c9-a9ad-8857339261c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:24.335312 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e9710-4afc-42c9-a9ad-8857339261c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:24.336036 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8e9710-4afc-42c9-a9ad-8857339261c9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:24.336342 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8e9710-4afc-42c9-a9ad-8857339261c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:24.337768 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8e9710-4afc-42c9-a9ad-8857339261c9 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:24.346210 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 138/279] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 10:13:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:24.347021 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e9710-4afc-42c9-a9ad-8857339261c9 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:24.428804 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b279901d-f1c9-43e6-90e3-798e92a7153b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:24.437746 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:24.446923 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e9710-4afc-42c9-a9ad-8857339261c9 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:24.462525 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c3602-f527-4757-bdc7-69e3d6277778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:24.463213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c3602-f527-4757-bdc7-69e3d6277778 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:24.463489 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c3602-f527-4757-bdc7-69e3d6277778 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:24.463755 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c3602-f527-4757-bdc7-69e3d6277778 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:24.464892 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030c3602-f527-4757-bdc7-69e3d6277778 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:24.466355 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030c3602-f527-4757-bdc7-69e3d6277778 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:24.468029 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 142/280] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:13:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:24.469285 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a8dddb-776c-4b73-ae45-2f8677a7be21 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:24.469557 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a8dddb-776c-4b73-ae45-2f8677a7be21 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:24.469830 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a8dddb-776c-4b73-ae45-2f8677a7be21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:24.481153 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e748c3-6294-4588-ab5d-e33122cc3876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:24.481835 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e748c3-6294-4588-ab5d-e33122cc3876 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:24.482123 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e748c3-6294-4588-ab5d-e33122cc3876 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:24.482396 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e748c3-6294-4588-ab5d-e33122cc3876 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:24.483152 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e748c3-6294-4588-ab5d-e33122cc3876 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:24.483544 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e748c3-6294-4588-ab5d-e33122cc3876 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:24.484864 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e748c3-6294-4588-ab5d-e33122cc3876 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:24.486964 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b1a8dddb-776c-4b73-ae45-2f8677a7be21 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:24.488141 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b1a8dddb-776c-4b73-ae45-2f8677a7be21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:24.493969 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 139/281] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 10:13:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:24.494772 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e748c3-6294-4588-ab5d-e33122cc3876 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:24.495820 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:24.555927 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 143/282] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:24.556738 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a8dddb-776c-4b73-ae45-2f8677a7be21 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:24.619701 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e748c3-6294-4588-ab5d-e33122cc3876 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:24.653034 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a8dddb-776c-4b73-ae45-2f8677a7be21 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:24.654012 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc9c06-e4b4-4b12-83d0-1fb0513b2a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:24.654659 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc9c06-e4b4-4b12-83d0-1fb0513b2a3f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:24.654967 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc9c06-e4b4-4b12-83d0-1fb0513b2a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:24.655341 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc9c06-e4b4-4b12-83d0-1fb0513b2a3f None admin] PATH_INFO: `/v3/users/399007ba69804ca7ac9376d1f5e728c2` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:24.668566 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bc9c06-e4b4-4b12-83d0-1fb0513b2a3f None admin] RBAC: Authorizing `identity:get_user(user_id=399007ba69804ca7ac9376d1f5e728c2)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:24.670371 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bc9c06-e4b4-4b12-83d0-1fb0513b2a3f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:24.672158 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 140/283] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:13:24 2022] GET /identity/v3/users/399007ba69804ca7ac9376d1f5e728c2 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:24.677005 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc9c06-e4b4-4b12-83d0-1fb0513b2a3f None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:24.687388 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:24.691661 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d43801-0ac1-4d40-b4e9-9b23f6e6aa2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:24.692293 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d43801-0ac1-4d40-b4e9-9b23f6e6aa2d None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:24.692570 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d43801-0ac1-4d40-b4e9-9b23f6e6aa2d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:24.692843 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d43801-0ac1-4d40-b4e9-9b23f6e6aa2d None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:24.693524 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d43801-0ac1-4d40-b4e9-9b23f6e6aa2d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:24.695002 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d43801-0ac1-4d40-b4e9-9b23f6e6aa2d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:24.711942 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:24.730935 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 144/284] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 5 10:13:24 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:13:24.730935 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d43801-0ac1-4d40-b4e9-9b23f6e6aa2d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:24.783729 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc9c06-e4b4-4b12-83d0-1fb0513b2a3f None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:24.836480 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a86de40-7f1f-4906-8a57-a309a24f1090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:24.837135 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86de40-7f1f-4906-8a57-a309a24f1090 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:24.837473 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86de40-7f1f-4906-8a57-a309a24f1090 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:24.837826 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86de40-7f1f-4906-8a57-a309a24f1090 None admin] PATH_INFO: `/v3/users/e2051851b02d431891a5a0ccd79b860f` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:24.839175 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a86de40-7f1f-4906-8a57-a309a24f1090 None admin] RBAC: Authorizing `identity:get_user(user_id=e2051851b02d431891a5a0ccd79b860f)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:24.840757 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a86de40-7f1f-4906-8a57-a309a24f1090 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:24.848255 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 141/285] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:13:24 2022] GET /identity/v3/users/e2051851b02d431891a5a0ccd79b860f => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:24.848255 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a86de40-7f1f-4906-8a57-a309a24f1090 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:24.865429 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d43801-0ac1-4d40-b4e9-9b23f6e6aa2d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:24.903741 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc78a45-83f4-411f-9d02-e525fee5abce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:24.904400 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc78a45-83f4-411f-9d02-e525fee5abce None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:24.904703 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc78a45-83f4-411f-9d02-e525fee5abce None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:24.905023 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc78a45-83f4-411f-9d02-e525fee5abce None admin] PATH_INFO: `/v3/roles/member` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:24.918916 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc78a45-83f4-411f-9d02-e525fee5abce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:24.924810 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-7cc78a45-83f4-411f-9d02-e525fee5abce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 10:13:24.924810 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:24.924810 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:24.924810 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:24.924810 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:24.924810 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:24.924810 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:24.924810 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:24.924810 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:24.924810 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:24.924810 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:24.924810 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:24.924810 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:24.924810 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:24.924810 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:24.924810 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:24.925679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:24.925679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:24.925679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:24.925679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:24.925679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:24.925679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:24.925679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:24.925679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:24.925679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:24.925679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:24.925679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:24.925679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:24.925679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:24.925679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:24.925679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:24.925679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:24.926478 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:24.926478 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:24.926478 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:24.926478 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:24.926478 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:24.926478 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:24.926478 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:24.926478 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:24.926478 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:24.926478 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:24.926478 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 10:13:24.926478 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:24.926478 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 145/286] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:13:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:24.927152 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc78a45-83f4-411f-9d02-e525fee5abce None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:24.951341 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:24.974603 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a86de40-7f1f-4906-8a57-a309a24f1090 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:25.005738 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:25.017539 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1064c-8d74-4148-a57b-80318fca53b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:25.018166 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1064c-8d74-4148-a57b-80318fca53b3 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:25.018442 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1064c-8d74-4148-a57b-80318fca53b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:25.018714 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1064c-8d74-4148-a57b-80318fca53b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:25.019546 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c1064c-8d74-4148-a57b-80318fca53b3 None admin] RBAC: Adding query filter params (role.id=c4ba94fa4bd34a09ba87d357142d9db8, user.id=399007ba69804ca7ac9376d1f5e728c2, scope.system=all) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:25.019850 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c1064c-8d74-4148-a57b-80318fca53b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:25.021293 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c1064c-8d74-4148-a57b-80318fca53b3 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:25.027701 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 142/287] 10.222.0.40 () {56 vars in 1308 bytes} [Fri Aug 5 10:13:24 2022] GET /identity/v3/role_assignments?user.id=399007ba69804ca7ac9376d1f5e728c2&scope.system=all&role.id=c4ba94fa4bd34a09ba87d357142d9db8 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:25.028594 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1064c-8d74-4148-a57b-80318fca53b3 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:25.058942 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc78a45-83f4-411f-9d02-e525fee5abce None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:25.102354 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b93b8d-d5cf-4a91-8669-c72ee4916425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:25.103087 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b93b8d-d5cf-4a91-8669-c72ee4916425 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:25.103367 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b93b8d-d5cf-4a91-8669-c72ee4916425 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:25.103642 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b93b8d-d5cf-4a91-8669-c72ee4916425 None admin] PATH_INFO: `/v3/projects/400a983b4cd947d487b7fa86a5f5c339` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:25.104944 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b93b8d-d5cf-4a91-8669-c72ee4916425 None admin] RBAC: Authorizing `identity:get_project(project_id=400a983b4cd947d487b7fa86a5f5c339)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:25.106459 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b93b8d-d5cf-4a91-8669-c72ee4916425 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:25.118166 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 146/288] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:13:24 2022] GET /identity/v3/projects/400a983b4cd947d487b7fa86a5f5c339 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:25.127215 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b93b8d-d5cf-4a91-8669-c72ee4916425 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:25.185397 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1064c-8d74-4148-a57b-80318fca53b3 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:25.260391 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75809383-0c9a-4fd4-97bd-450748ccbc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:25.261738 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75809383-0c9a-4fd4-97bd-450748ccbc38 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:25.262096 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75809383-0c9a-4fd4-97bd-450748ccbc38 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:25.262435 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75809383-0c9a-4fd4-97bd-450748ccbc38 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:25.263264 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75809383-0c9a-4fd4-97bd-450748ccbc38 None admin] RBAC: Adding query filter params (name=member) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:25.263636 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75809383-0c9a-4fd4-97bd-450748ccbc38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:25.266688 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75809383-0c9a-4fd4-97bd-450748ccbc38 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:25.283049 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 143/289] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 10:13:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:25.285773 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75809383-0c9a-4fd4-97bd-450748ccbc38 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:25.300894 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:25.331955 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b93b8d-d5cf-4a91-8669-c72ee4916425 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:25.384462 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:25.400455 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40134123-c9b8-46ce-8d48-7cef6cb80ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:25.401142 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40134123-c9b8-46ce-8d48-7cef6cb80ecd None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:25.401418 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40134123-c9b8-46ce-8d48-7cef6cb80ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:25.401689 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40134123-c9b8-46ce-8d48-7cef6cb80ecd None admin] PATH_INFO: `/v3/projects/400a983b4cd947d487b7fa86a5f5c339/users/e2051851b02d431891a5a0ccd79b860f/roles/cf5de414a035475b8b75626cb19a2de2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:25.405366 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40134123-c9b8-46ce-8d48-7cef6cb80ecd None admin] RBAC: Authorizing `identity:create_grant(project_id=400a983b4cd947d487b7fa86a5f5c339, user_id=e2051851b02d431891a5a0ccd79b860f, role_id=cf5de414a035475b8b75626cb19a2de2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:25.406884 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40134123-c9b8-46ce-8d48-7cef6cb80ecd None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:25.438706 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 147/290] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 10:13:25 2022] PUT /identity/v3/projects/400a983b4cd947d487b7fa86a5f5c339/users/e2051851b02d431891a5a0ccd79b860f/roles/cf5de414a035475b8b75626cb19a2de2 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:13:25.447976 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75809383-0c9a-4fd4-97bd-450748ccbc38 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:25.496227 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e796b-18f2-403a-9194-58d8a509fc02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:25.496907 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e796b-18f2-403a-9194-58d8a509fc02 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:25.497187 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e796b-18f2-403a-9194-58d8a509fc02 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:25.497520 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e796b-18f2-403a-9194-58d8a509fc02 None admin] PATH_INFO: `/v3/users/b17ad37d9c5d445990fce41f2172124d` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:25.498790 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e796b-18f2-403a-9194-58d8a509fc02 None admin] RBAC: Authorizing `identity:get_user(user_id=b17ad37d9c5d445990fce41f2172124d)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:25.500563 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e796b-18f2-403a-9194-58d8a509fc02 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:25.502278 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 144/291] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:13:25 2022] GET /identity/v3/users/b17ad37d9c5d445990fce41f2172124d => generated 297 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:25.508567 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40134123-c9b8-46ce-8d48-7cef6cb80ecd None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:25.662848 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40134123-c9b8-46ce-8d48-7cef6cb80ecd None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:25.704365 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1233359-0fea-4db5-b0cf-3a979894608b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:25.705007 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1233359-0fea-4db5-b0cf-3a979894608b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:25.705293 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1233359-0fea-4db5-b0cf-3a979894608b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:25.705568 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1233359-0fea-4db5-b0cf-3a979894608b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:25.772556 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:25.793541 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1233359-0fea-4db5-b0cf-3a979894608b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:25.795484 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1233359-0fea-4db5-b0cf-3a979894608b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:25.914558 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 148/292] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:13:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 10:13:25.920308 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e796b-18f2-403a-9194-58d8a509fc02 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:26.017544 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:26.042619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e796b-18f2-403a-9194-58d8a509fc02 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:26.077627 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f82a572-3fc5-4a34-8f1e-a6e9ff111157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:26.078277 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f82a572-3fc5-4a34-8f1e-a6e9ff111157 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:26.078555 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f82a572-3fc5-4a34-8f1e-a6e9ff111157 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:26.078822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f82a572-3fc5-4a34-8f1e-a6e9ff111157 None admin] PATH_INFO: `/v3/projects/400a983b4cd947d487b7fa86a5f5c339` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:26.080026 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f82a572-3fc5-4a34-8f1e-a6e9ff111157 None admin] RBAC: Authorizing `identity:get_project(project_id=400a983b4cd947d487b7fa86a5f5c339)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:26.081463 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f82a572-3fc5-4a34-8f1e-a6e9ff111157 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:26.083350 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 145/293] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:13:25 2022] GET /identity/v3/projects/400a983b4cd947d487b7fa86a5f5c339 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:26.089272 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1233359-0fea-4db5-b0cf-3a979894608b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:26.230226 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:26.248722 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1233359-0fea-4db5-b0cf-3a979894608b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:26.304818 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5960e043-3dfe-4ddd-b806-d4562ec10645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:26.305458 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5960e043-3dfe-4ddd-b806-d4562ec10645 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:26.305739 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5960e043-3dfe-4ddd-b806-d4562ec10645 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:26.306011 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5960e043-3dfe-4ddd-b806-d4562ec10645 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:26.306805 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5960e043-3dfe-4ddd-b806-d4562ec10645 None admin] RBAC: Adding query filter params (role.id=c4ba94fa4bd34a09ba87d357142d9db8, scope.project.id=400a983b4cd947d487b7fa86a5f5c339, user.id=b17ad37d9c5d445990fce41f2172124d) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:26.307158 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5960e043-3dfe-4ddd-b806-d4562ec10645 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:26.308627 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5960e043-3dfe-4ddd-b806-d4562ec10645 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:26.315698 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 149/294] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:13:26 2022] GET /identity/v3/role_assignments?user.id=b17ad37d9c5d445990fce41f2172124d&scope.project.id=400a983b4cd947d487b7fa86a5f5c339&role.id=c4ba94fa4bd34a09ba87d357142d9db8 => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:27.145045 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b20b7-9360-4e91-a936-603cc03709f5 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:27.145578 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b20b7-9360-4e91-a936-603cc03709f5 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:27.145578 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b20b7-9360-4e91-a936-603cc03709f5 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:27.147077 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 146/295] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:27.152808 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dae89c-6568-4d58-bb47-dd4a75ec91e5 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:27.153285 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dae89c-6568-4d58-bb47-dd4a75ec91e5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:27.153357 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dae89c-6568-4d58-bb47-dd4a75ec91e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:27.178444 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-21dae89c-6568-4d58-bb47-dd4a75ec91e5 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:27.180009 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-21dae89c-6568-4d58-bb47-dd4a75ec91e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:27.247373 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 150/296] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:27.270579 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b20b7-9360-4e91-a936-603cc03709f5 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:27.276495 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adaf0ea-ae05-4075-815d-ce1c5cd63ba1 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:27.276778 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adaf0ea-ae05-4075-815d-ce1c5cd63ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:27.277049 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adaf0ea-ae05-4075-815d-ce1c5cd63ba1 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:27.278422 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 151/297] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:27.283999 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c230486-bef3-4f2c-9228-34d04f965597 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:27.284228 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c230486-bef3-4f2c-9228-34d04f965597 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:27.284450 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c230486-bef3-4f2c-9228-34d04f965597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:27.303677 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8c230486-bef3-4f2c-9228-34d04f965597 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:27.304871 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8c230486-bef3-4f2c-9228-34d04f965597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:27.377958 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b20b7-9360-4e91-a936-603cc03709f5 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:27.378604 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 152/298] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 10:13:27.400275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c230486-bef3-4f2c-9228-34d04f965597 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:27.412102 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:27.413856 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc848a0-5531-4407-a790-a554d73a0df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:27.414723 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc848a0-5531-4407-a790-a554d73a0df9 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:27.415157 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc848a0-5531-4407-a790-a554d73a0df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:27.415405 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc848a0-5531-4407-a790-a554d73a0df9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:27.427841 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc848a0-5531-4407-a790-a554d73a0df9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:27.433646 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-6dc848a0-5531-4407-a790-a554d73a0df9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 10:13:27.433646 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:27.433646 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:27.433646 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:27.433646 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:27.433646 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:27.433646 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:27.433646 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:27.433646 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:27.433646 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:27.433646 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:27.433646 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:27.433646 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:27.433646 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:27.433646 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:27.433646 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:27.434517 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:27.434517 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:27.434517 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:27.434517 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:27.434517 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:27.434517 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:27.434517 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:27.434517 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:27.434517 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:27.434517 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:27.434517 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:27.434517 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:27.434517 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:27.434517 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:27.434517 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:27.434517 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:27.435333 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:27.435333 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:27.435333 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:27.435333 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:27.435333 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:27.435333 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:27.435333 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:27.435333 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:27.435333 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:27.435333 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:27.435333 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 10:13:27.435333 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:13:27.435333 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 147/299] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:13:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:27.440837 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc848a0-5531-4407-a790-a554d73a0df9 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:27.443861 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:27.511890 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c230486-bef3-4f2c-9228-34d04f965597 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:27.544499 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc848a0-5531-4407-a790-a554d73a0df9 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:27.547864 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6adf0-f1eb-4728-a828-2b4944bbc3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:27.548524 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6adf0-f1eb-4728-a828-2b4944bbc3c4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:27.548798 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6adf0-f1eb-4728-a828-2b4944bbc3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:27.549065 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6adf0-f1eb-4728-a828-2b4944bbc3c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:27.550207 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c6adf0-f1eb-4728-a828-2b4944bbc3c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:27.551690 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c6adf0-f1eb-4728-a828-2b4944bbc3c4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:27.554964 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 153/300] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:13:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:13:27.554964 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b87612-e53d-46b1-87d4-0a84a0532618 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:27.554964 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b87612-e53d-46b1-87d4-0a84a0532618 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:27.555230 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b87612-e53d-46b1-87d4-0a84a0532618 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:27.559066 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 154/301] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:27.559066 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b87612-e53d-46b1-87d4-0a84a0532618 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:27.589182 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b394a906-e702-4f3c-9f02-c6abbcb76b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:27.589799 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394a906-e702-4f3c-9f02-c6abbcb76b3f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:27.590077 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394a906-e702-4f3c-9f02-c6abbcb76b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:27.590345 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394a906-e702-4f3c-9f02-c6abbcb76b3f None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:27.591100 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b394a906-e702-4f3c-9f02-c6abbcb76b3f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:27.591400 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b394a906-e702-4f3c-9f02-c6abbcb76b3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:27.592882 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b394a906-e702-4f3c-9f02-c6abbcb76b3f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:27.601127 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 148/302] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 10:13:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:27.602280 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5e9b92-018a-490c-a118-41b97f766e2d None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:27.602560 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5e9b92-018a-490c-a118-41b97f766e2d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:27.602831 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5e9b92-018a-490c-a118-41b97f766e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:27.633154 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ee5e9b92-018a-490c-a118-41b97f766e2d None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:27.634330 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ee5e9b92-018a-490c-a118-41b97f766e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:27.649435 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:27.675797 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b87612-e53d-46b1-87d4-0a84a0532618 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:27.695577 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:27.705367 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 149/303] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:27.715116 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5e9b92-018a-490c-a118-41b97f766e2d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:27.718493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e650f3-ba4c-4c4b-b32d-b1566ef010fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:27.719178 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e650f3-ba4c-4c4b-b32d-b1566ef010fc None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:27.719500 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e650f3-ba4c-4c4b-b32d-b1566ef010fc None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:27.719769 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e650f3-ba4c-4c4b-b32d-b1566ef010fc None admin] PATH_INFO: `/v3/groups` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:27.720629 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e650f3-ba4c-4c4b-b32d-b1566ef010fc None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:27.722173 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e650f3-ba4c-4c4b-b32d-b1566ef010fc None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:27.740676 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 155/304] 10.222.0.40 () {60 vars in 1134 bytes} [Fri Aug 5 10:13:27 2022] POST /identity/v3/groups => generated 229 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:13:27.741938 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e650f3-ba4c-4c4b-b32d-b1566ef010fc None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:27.831346 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5e9b92-018a-490c-a118-41b97f766e2d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:27.870658 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569cc901-ed56-45fa-944f-464b4d552dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:27.871356 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569cc901-ed56-45fa-944f-464b4d552dbb None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:27.871637 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569cc901-ed56-45fa-944f-464b4d552dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:27.871916 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569cc901-ed56-45fa-944f-464b4d552dbb None admin] PATH_INFO: `/v3/users/79c11a00e4994d83919c7f59026b4a12` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:27.872037 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e650f3-ba4c-4c4b-b32d-b1566ef010fc None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:27.891290 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569cc901-ed56-45fa-944f-464b4d552dbb None admin] RBAC: Authorizing `identity:get_user(user_id=79c11a00e4994d83919c7f59026b4a12)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:27.893091 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569cc901-ed56-45fa-944f-464b4d552dbb None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:27.900054 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 150/305] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:13:27 2022] GET /identity/v3/users/79c11a00e4994d83919c7f59026b4a12 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:27.900054 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f201f-3f5a-49d7-9f1a-4120626394f6 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:27.900054 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f201f-3f5a-49d7-9f1a-4120626394f6 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:27.900054 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f201f-3f5a-49d7-9f1a-4120626394f6 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:27.903534 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 151/306] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:27.903992 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f201f-3f5a-49d7-9f1a-4120626394f6 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:27.915953 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a721f499-1878-4821-ac22-ed8b98ba55ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:27.917305 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a721f499-1878-4821-ac22-ed8b98ba55ad None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:27.917592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a721f499-1878-4821-ac22-ed8b98ba55ad None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:27.917874 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a721f499-1878-4821-ac22-ed8b98ba55ad None admin] PATH_INFO: `/v3/roles/member` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:27.927092 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:27.934163 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a721f499-1878-4821-ac22-ed8b98ba55ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:27.940203 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-a721f499-1878-4821-ac22-ed8b98ba55ad None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 10:13:27.940203 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:27.940203 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:27.940203 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:27.940203 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:27.940203 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:27.940203 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:27.940203 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:27.940203 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:27.940203 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:27.940203 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:27.940203 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:27.940203 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:27.940203 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:27.940203 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:27.940203 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:27.941078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:27.941078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:27.941078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:27.941078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:27.941078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:27.941078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:27.941078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:27.941078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:27.941078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:27.941078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:27.941078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:27.941078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:27.941078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:27.941078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:27.941078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:27.941078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:27.941890 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:27.941890 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:27.941890 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:27.941890 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:27.941890 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:27.941890 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:27.941890 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:27.941890 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:27.941890 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:27.941890 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:27.941890 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 10:13:27.941890 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:27.941890 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 156/307] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:13:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:27.944083 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c811500-d8eb-437f-baea-e4f286d023b7 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:27.947335 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c811500-d8eb-437f-baea-e4f286d023b7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:27.947619 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c811500-d8eb-437f-baea-e4f286d023b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:27.967396 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6c811500-d8eb-437f-baea-e4f286d023b7 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:27.968834 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6c811500-d8eb-437f-baea-e4f286d023b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:27.971427 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:28.028161 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f201f-3f5a-49d7-9f1a-4120626394f6 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:28.037449 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 157/308] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:28.038483 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c811500-d8eb-437f-baea-e4f286d023b7 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:28.072037 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1661069-ca22-470b-b388-07534a98d93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:28.072762 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1661069-ca22-470b-b388-07534a98d93c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:28.073048 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1661069-ca22-470b-b388-07534a98d93c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:28.073320 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1661069-ca22-470b-b388-07534a98d93c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:28.074123 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1661069-ca22-470b-b388-07534a98d93c None admin] RBAC: Adding query filter params (role.id=cf5de414a035475b8b75626cb19a2de2, user.id=79c11a00e4994d83919c7f59026b4a12, scope.system=all) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:28.074435 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1661069-ca22-470b-b388-07534a98d93c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:28.075924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1661069-ca22-470b-b388-07534a98d93c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:28.081105 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 152/309] 10.222.0.40 () {56 vars in 1308 bytes} [Fri Aug 5 10:13:27 2022] GET /identity/v3/role_assignments?user.id=79c11a00e4994d83919c7f59026b4a12&scope.system=all&role.id=cf5de414a035475b8b75626cb19a2de2 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:28.081901 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1661069-ca22-470b-b388-07534a98d93c None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:28.193323 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c811500-d8eb-437f-baea-e4f286d023b7 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:28.212458 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1661069-ca22-470b-b388-07534a98d93c None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:28.231263 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:28.237905 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:28.247671 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b80d75-f3af-4ca7-bcfc-979f8265d366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:28.248493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b80d75-f3af-4ca7-bcfc-979f8265d366 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:28.248601 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b80d75-f3af-4ca7-bcfc-979f8265d366 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:28.249015 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b80d75-f3af-4ca7-bcfc-979f8265d366 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:28.250419 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b80d75-f3af-4ca7-bcfc-979f8265d366 None admin] RBAC: Adding query filter params (name=member) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:28.250731 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b80d75-f3af-4ca7-bcfc-979f8265d366 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:28.252199 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b80d75-f3af-4ca7-bcfc-979f8265d366 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:28.259052 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15389c28-a23e-41e8-96c7-049c94be3a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:28.260326 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15389c28-a23e-41e8-96c7-049c94be3a39 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:28.260800 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15389c28-a23e-41e8-96c7-049c94be3a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:28.261259 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15389c28-a23e-41e8-96c7-049c94be3a39 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:28.267026 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 158/310] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 10:13:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:28.268374 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b80d75-f3af-4ca7-bcfc-979f8265d366 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:28.275832 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15389c28-a23e-41e8-96c7-049c94be3a39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:28.283125 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-15389c28-a23e-41e8-96c7-049c94be3a39 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 10:13:28.283125 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:28.283125 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:28.283125 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:28.283125 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:28.283125 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:28.283125 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:28.283125 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:28.283125 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:28.283125 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:28.283125 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:28.283125 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:28.283125 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:28.283125 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:28.283125 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:28.283125 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:28.284020 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:28.284020 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:28.284020 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:28.284020 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:28.284020 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:28.284020 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:28.284020 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:28.284020 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:28.284020 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:28.284020 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:28.284020 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:28.284020 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:28.284020 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:28.284020 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:28.284020 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:28.284020 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:28.284810 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:28.284810 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:28.284810 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:28.284810 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:28.284810 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:28.284810 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:28.284810 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:28.284810 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:28.284810 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:28.284810 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:28.284810 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 10:13:28.284810 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:13:28.287484 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 153/311] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:13:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:28.298960 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15389c28-a23e-41e8-96c7-049c94be3a39 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:28.380028 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b80d75-f3af-4ca7-bcfc-979f8265d366 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:28.418756 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a85405-91e6-47b5-8f85-bc60d8273917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:28.419487 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a85405-91e6-47b5-8f85-bc60d8273917 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:28.420110 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a85405-91e6-47b5-8f85-bc60d8273917 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:28.420376 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a85405-91e6-47b5-8f85-bc60d8273917 None admin] PATH_INFO: `/v3/users/399007ba69804ca7ac9376d1f5e728c2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:28.421829 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a85405-91e6-47b5-8f85-bc60d8273917 None admin] RBAC: Authorizing `identity:get_user(user_id=399007ba69804ca7ac9376d1f5e728c2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:28.423332 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a85405-91e6-47b5-8f85-bc60d8273917 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:28.429811 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 159/312] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:13:28 2022] GET /identity/v3/users/399007ba69804ca7ac9376d1f5e728c2 => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:28.431220 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a85405-91e6-47b5-8f85-bc60d8273917 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:28.436337 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15389c28-a23e-41e8-96c7-049c94be3a39 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:28.476440 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cbafbf-5092-4539-93a4-1ed907c66e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:28.477082 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cbafbf-5092-4539-93a4-1ed907c66e3e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:28.477361 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cbafbf-5092-4539-93a4-1ed907c66e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:28.478542 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cbafbf-5092-4539-93a4-1ed907c66e3e None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:28.479388 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cbafbf-5092-4539-93a4-1ed907c66e3e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:28.479697 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cbafbf-5092-4539-93a4-1ed907c66e3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:28.481154 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cbafbf-5092-4539-93a4-1ed907c66e3e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:28.497766 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 154/313] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 10:13:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:13:28.500096 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cbafbf-5092-4539-93a4-1ed907c66e3e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:28.506684 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:28.545580 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:28.550953 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a85405-91e6-47b5-8f85-bc60d8273917 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:28.586626 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35605c13-1713-48e1-9242-02511b693b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:28.587341 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35605c13-1713-48e1-9242-02511b693b59 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:28.587624 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35605c13-1713-48e1-9242-02511b693b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:28.588132 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35605c13-1713-48e1-9242-02511b693b59 None admin] PATH_INFO: `/v3/system/users/399007ba69804ca7ac9376d1f5e728c2/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:28.589360 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35605c13-1713-48e1-9242-02511b693b59 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=399007ba69804ca7ac9376d1f5e728c2, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:28.590744 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35605c13-1713-48e1-9242-02511b693b59 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:28.603852 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 160/314] 10.222.0.40 () {58 vars in 1334 bytes} [Fri Aug 5 10:13:28 2022] PUT /identity/v3/system/users/399007ba69804ca7ac9376d1f5e728c2/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:13:28.618697 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cbafbf-5092-4539-93a4-1ed907c66e3e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:28.661141 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd00ced-99dc-49ea-803b-959edf8ff613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:28.662099 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd00ced-99dc-49ea-803b-959edf8ff613 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:28.662546 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd00ced-99dc-49ea-803b-959edf8ff613 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:28.663024 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd00ced-99dc-49ea-803b-959edf8ff613 None admin] PATH_INFO: `/v3/users/e2051851b02d431891a5a0ccd79b860f` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:28.664649 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd00ced-99dc-49ea-803b-959edf8ff613 None admin] RBAC: Authorizing `identity:get_user(user_id=e2051851b02d431891a5a0ccd79b860f)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:28.666366 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd00ced-99dc-49ea-803b-959edf8ff613 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:28.668714 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 155/315] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:13:28 2022] GET /identity/v3/users/e2051851b02d431891a5a0ccd79b860f => generated 297 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:13:28.675163 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35605c13-1713-48e1-9242-02511b693b59 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:28.783173 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35605c13-1713-48e1-9242-02511b693b59 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:28.807224 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:28.821339 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c522893-0386-4bcb-ab00-76150945355e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:28.821969 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c522893-0386-4bcb-ab00-76150945355e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:28.822245 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c522893-0386-4bcb-ab00-76150945355e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:28.822511 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c522893-0386-4bcb-ab00-76150945355e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:28.894282 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c522893-0386-4bcb-ab00-76150945355e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:28.895789 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c522893-0386-4bcb-ab00-76150945355e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:29.006112 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 161/316] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:13:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 10:13:29.012228 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd00ced-99dc-49ea-803b-959edf8ff613 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:29.116040 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd00ced-99dc-49ea-803b-959edf8ff613 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:29.127287 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:29.151642 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7280985-7c78-4fcc-88ba-7b76c16e4000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:29.152306 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7280985-7c78-4fcc-88ba-7b76c16e4000 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:29.152588 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7280985-7c78-4fcc-88ba-7b76c16e4000 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:29.152864 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7280985-7c78-4fcc-88ba-7b76c16e4000 None admin] PATH_INFO: `/v3/projects/400a983b4cd947d487b7fa86a5f5c339` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:29.154077 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7280985-7c78-4fcc-88ba-7b76c16e4000 None admin] RBAC: Authorizing `identity:get_project(project_id=400a983b4cd947d487b7fa86a5f5c339)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:29.155618 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7280985-7c78-4fcc-88ba-7b76c16e4000 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:29.157624 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 156/317] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:13:29 2022] GET /identity/v3/projects/400a983b4cd947d487b7fa86a5f5c339 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:29.163486 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c522893-0386-4bcb-ab00-76150945355e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:29.188084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:29.268771 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c522893-0386-4bcb-ab00-76150945355e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:29.303778 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4272a386-54ce-4b44-bec7-2942a43362b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:29.304407 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4272a386-54ce-4b44-bec7-2942a43362b6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:29.304688 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4272a386-54ce-4b44-bec7-2942a43362b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:29.304963 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4272a386-54ce-4b44-bec7-2942a43362b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:29.305764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4272a386-54ce-4b44-bec7-2942a43362b6 None admin] RBAC: Adding query filter params (role.id=cf5de414a035475b8b75626cb19a2de2, scope.project.id=400a983b4cd947d487b7fa86a5f5c339, user.id=e2051851b02d431891a5a0ccd79b860f) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:29.306074 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4272a386-54ce-4b44-bec7-2942a43362b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:29.307569 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4272a386-54ce-4b44-bec7-2942a43362b6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:29.314076 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 162/318] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:13:29 2022] GET /identity/v3/role_assignments?user.id=e2051851b02d431891a5a0ccd79b860f&scope.project.id=400a983b4cd947d487b7fa86a5f5c339&role.id=cf5de414a035475b8b75626cb19a2de2 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:29.812766 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df6f02f-0414-4443-b4d8-35b2778eeb6d None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:29.813313 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df6f02f-0414-4443-b4d8-35b2778eeb6d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:29.813313 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df6f02f-0414-4443-b4d8-35b2778eeb6d None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:29.814647 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 157/319] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:29.820192 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece9abf-92b9-4222-aa55-4e7aecfbe11d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:29.820481 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece9abf-92b9-4222-aa55-4e7aecfbe11d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:29.820713 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece9abf-92b9-4222-aa55-4e7aecfbe11d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:29.839152 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-aece9abf-92b9-4222-aa55-4e7aecfbe11d None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:29.840330 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-aece9abf-92b9-4222-aa55-4e7aecfbe11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:29.910703 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 163/320] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:29.932641 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df6f02f-0414-4443-b4d8-35b2778eeb6d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:30.035661 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df6f02f-0414-4443-b4d8-35b2778eeb6d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:30.070126 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0aa3ad-5717-4b3c-aa3a-a21cbf1bc7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:30.070703 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0aa3ad-5717-4b3c-aa3a-a21cbf1bc7be None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:30.071050 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0aa3ad-5717-4b3c-aa3a-a21cbf1bc7be None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:30.071360 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0aa3ad-5717-4b3c-aa3a-a21cbf1bc7be None admin] PATH_INFO: `/v3/domains/default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:30.072540 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0aa3ad-5717-4b3c-aa3a-a21cbf1bc7be None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:30.074017 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0aa3ad-5717-4b3c-aa3a-a21cbf1bc7be None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:30.075812 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 158/321] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:13:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:30.082044 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece9abf-92b9-4222-aa55-4e7aecfbe11d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:30.100410 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:30.172921 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c115d1-7503-40be-9360-72acf4b38eaa None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:30.173250 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c115d1-7503-40be-9360-72acf4b38eaa None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:30.173557 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c115d1-7503-40be-9360-72acf4b38eaa None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:30.174953 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 159/322] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 10:13:30.180059 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9197d66a-ba83-4aab-af33-238c9fc73d8c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:30.180381 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9197d66a-ba83-4aab-af33-238c9fc73d8c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:30.180693 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9197d66a-ba83-4aab-af33-238c9fc73d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:30.192172 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece9abf-92b9-4222-aa55-4e7aecfbe11d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:30.199238 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-9197d66a-ba83-4aab-af33-238c9fc73d8c None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:30.200447 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9197d66a-ba83-4aab-af33-238c9fc73d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:30.231802 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0880156-b158-4cbd-ae67-7b8f2ff0c418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:30.232429 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0880156-b158-4cbd-ae67-7b8f2ff0c418 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:30.232734 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0880156-b158-4cbd-ae67-7b8f2ff0c418 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:30.233031 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0880156-b158-4cbd-ae67-7b8f2ff0c418 None admin] PATH_INFO: `/v3/groups` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:30.233719 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0880156-b158-4cbd-ae67-7b8f2ff0c418 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:30.235223 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0880156-b158-4cbd-ae67-7b8f2ff0c418 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:30.239112 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:30.251708 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 164/323] 10.222.0.40 () {60 vars in 1134 bytes} [Fri Aug 5 10:13:30 2022] POST /identity/v3/groups => generated 226 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:13:30.279021 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 160/324] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 10:13:30.298724 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0880156-b158-4cbd-ae67-7b8f2ff0c418 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:30.386432 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:30.415222 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0880156-b158-4cbd-ae67-7b8f2ff0c418 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:30.450502 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dbf4c4-a3af-4726-8109-940111b2ca8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:30.451262 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dbf4c4-a3af-4726-8109-940111b2ca8f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:30.451543 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dbf4c4-a3af-4726-8109-940111b2ca8f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:30.451914 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dbf4c4-a3af-4726-8109-940111b2ca8f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:30.462429 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dbf4c4-a3af-4726-8109-940111b2ca8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:30.467878 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-55dbf4c4-a3af-4726-8109-940111b2ca8f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 10:13:30.467878 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:30.467878 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:30.467878 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:30.467878 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:30.467878 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:30.467878 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:30.467878 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:30.467878 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:30.467878 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:30.467878 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:30.467878 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:30.467878 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:30.467878 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:30.467878 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:30.467878 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:30.468764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:30.468764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:30.468764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:30.468764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:30.468764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:30.468764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:30.468764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:30.468764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:30.468764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:30.468764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:30.468764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:30.468764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:30.468764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:30.468764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:30.468764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:30.468764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:30.469538 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:30.469538 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:30.469538 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:30.469538 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:30.469538 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:30.469538 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:30.469538 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:30.469538 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:30.469538 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:30.469538 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:30.469538 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 10:13:30.469538 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:30.469538 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 165/325] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:13:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:30.475196 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9197d66a-ba83-4aab-af33-238c9fc73d8c None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:30.580800 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9197d66a-ba83-4aab-af33-238c9fc73d8c None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:30.618616 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b8b764-b714-4531-bada-bf696cbeaf86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:30.619389 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b8b764-b714-4531-bada-bf696cbeaf86 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:30.619705 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b8b764-b714-4531-bada-bf696cbeaf86 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:30.620012 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b8b764-b714-4531-bada-bf696cbeaf86 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:30.620779 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b8b764-b714-4531-bada-bf696cbeaf86 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:30.621107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b8b764-b714-4531-bada-bf696cbeaf86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:30.622495 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b8b764-b714-4531-bada-bf696cbeaf86 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:30.633475 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 161/326] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 10:13:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:30.639439 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dbf4c4-a3af-4726-8109-940111b2ca8f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:30.666464 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960afac-42f3-46fe-8850-19929d477781 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:30.666797 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960afac-42f3-46fe-8850-19929d477781 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:30.667172 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960afac-42f3-46fe-8850-19929d477781 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:30.668543 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 162/327] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:30.674246 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b9264-f082-4713-8c04-4912489e69e9 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:30.674565 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b9264-f082-4713-8c04-4912489e69e9 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:30.674873 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b9264-f082-4713-8c04-4912489e69e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:30.700299 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ab1b9264-f082-4713-8c04-4912489e69e9 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:30.701864 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ab1b9264-f082-4713-8c04-4912489e69e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:30.735082 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:30.791565 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dbf4c4-a3af-4726-8109-940111b2ca8f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:30.792891 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 163/328] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:30.816813 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1b9264-f082-4713-8c04-4912489e69e9 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:30.835363 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07bfa42-ddde-41b6-a9d3-30bcb9366e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:30.836385 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07bfa42-ddde-41b6-a9d3-30bcb9366e09 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:30.836868 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07bfa42-ddde-41b6-a9d3-30bcb9366e09 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:30.837389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07bfa42-ddde-41b6-a9d3-30bcb9366e09 None admin] PATH_INFO: `/v3/users/79c11a00e4994d83919c7f59026b4a12` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:30.838801 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07bfa42-ddde-41b6-a9d3-30bcb9366e09 None admin] RBAC: Authorizing `identity:get_user(user_id=79c11a00e4994d83919c7f59026b4a12)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:30.840370 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07bfa42-ddde-41b6-a9d3-30bcb9366e09 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:30.842419 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 166/329] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:13:30 2022] GET /identity/v3/users/79c11a00e4994d83919c7f59026b4a12 => generated 293 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:13:30.848258 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07bfa42-ddde-41b6-a9d3-30bcb9366e09 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:30.852319 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:30.980712 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1b9264-f082-4713-8c04-4912489e69e9 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:30.985937 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07bfa42-ddde-41b6-a9d3-30bcb9366e09 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:31.024272 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea4ae6-53a7-4b5c-8bde-e122c498cb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:31.024974 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea4ae6-53a7-4b5c-8bde-e122c498cb71 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:31.025276 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea4ae6-53a7-4b5c-8bde-e122c498cb71 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:31.025589 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea4ae6-53a7-4b5c-8bde-e122c498cb71 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:31.028377 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc6a0a7-027c-4c84-a36b-950009b56a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:31.029559 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6a0a7-027c-4c84-a36b-950009b56a25 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:31.030147 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6a0a7-027c-4c84-a36b-950009b56a25 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:31.030611 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6a0a7-027c-4c84-a36b-950009b56a25 None admin] PATH_INFO: `/v3/system/users/79c11a00e4994d83919c7f59026b4a12/roles/cf5de414a035475b8b75626cb19a2de2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:31.032599 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc6a0a7-027c-4c84-a36b-950009b56a25 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=79c11a00e4994d83919c7f59026b4a12, role_id=cf5de414a035475b8b75626cb19a2de2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:31.034514 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc6a0a7-027c-4c84-a36b-950009b56a25 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:31.040115 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ea4ae6-53a7-4b5c-8bde-e122c498cb71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:31.045949 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-86ea4ae6-53a7-4b5c-8bde-e122c498cb71 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 10:13:31.045949 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:31.045949 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:31.045949 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:31.045949 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:31.045949 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:31.045949 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:31.045949 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:31.045949 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:31.045949 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:31.045949 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:31.045949 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:31.045949 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:31.045949 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:31.045949 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:31.045949 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:31.047084 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:31.047084 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:31.047084 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:31.047084 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:31.047084 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:31.047084 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:31.047084 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:31.047084 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:31.047084 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:31.047084 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:31.047084 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:31.047084 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:31.047084 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:31.047084 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:31.047084 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:31.047084 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:31.048662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:31.048662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:31.048662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:31.048662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:31.048662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:31.048662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:31.048662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:31.048662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:31.048662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:31.048662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:31.048662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 10:13:31.048662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:13:31.048662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 164/330] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:13:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:13:31.049306 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 167/331] 10.222.0.40 () {58 vars in 1334 bytes} [Fri Aug 5 10:13:30 2022] PUT /identity/v3/system/users/79c11a00e4994d83919c7f59026b4a12/roles/cf5de414a035475b8b75626cb19a2de2 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:13:31.060361 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea4ae6-53a7-4b5c-8bde-e122c498cb71 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:31.096519 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:31.182630 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea4ae6-53a7-4b5c-8bde-e122c498cb71 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:31.221243 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de7c8fd-0dfa-47ac-aea7-3ecd0f5a199e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:31.222344 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7c8fd-0dfa-47ac-aea7-3ecd0f5a199e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:31.222812 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7c8fd-0dfa-47ac-aea7-3ecd0f5a199e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:31.223325 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7c8fd-0dfa-47ac-aea7-3ecd0f5a199e None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:31.224266 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de7c8fd-0dfa-47ac-aea7-3ecd0f5a199e None admin] RBAC: Adding query filter params (name=member) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:31.224780 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de7c8fd-0dfa-47ac-aea7-3ecd0f5a199e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:31.226538 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de7c8fd-0dfa-47ac-aea7-3ecd0f5a199e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:31.240300 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 165/332] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 10:13:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:31.245856 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc6a0a7-027c-4c84-a36b-950009b56a25 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:31.325465 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:31.365886 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc6a0a7-027c-4c84-a36b-950009b56a25 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:31.402635 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7b5a2-1d07-4e84-bffc-4ca7ba53657a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:31.403644 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7b5a2-1d07-4e84-bffc-4ca7ba53657a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:31.404132 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7b5a2-1d07-4e84-bffc-4ca7ba53657a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:31.404588 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7b5a2-1d07-4e84-bffc-4ca7ba53657a None admin] PATH_INFO: `/v3/users/399007ba69804ca7ac9376d1f5e728c2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:31.406218 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f7b5a2-1d07-4e84-bffc-4ca7ba53657a None admin] RBAC: Authorizing `identity:get_user(user_id=399007ba69804ca7ac9376d1f5e728c2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:31.407903 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f7b5a2-1d07-4e84-bffc-4ca7ba53657a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:31.410076 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 168/333] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:13:31 2022] GET /identity/v3/users/399007ba69804ca7ac9376d1f5e728c2 => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:31.420410 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de7c8fd-0dfa-47ac-aea7-3ecd0f5a199e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:31.516182 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:31.532234 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de7c8fd-0dfa-47ac-aea7-3ecd0f5a199e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:31.577089 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78709c0f-1ff6-4d8a-a427-9edfe53afc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:31.578121 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78709c0f-1ff6-4d8a-a427-9edfe53afc54 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:31.578612 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78709c0f-1ff6-4d8a-a427-9edfe53afc54 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:31.579091 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78709c0f-1ff6-4d8a-a427-9edfe53afc54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:31.580120 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78709c0f-1ff6-4d8a-a427-9edfe53afc54 None admin] RBAC: Adding query filter params (role.id=c4ba94fa4bd34a09ba87d357142d9db8, user.id=399007ba69804ca7ac9376d1f5e728c2, scope.system=all) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:31.580668 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78709c0f-1ff6-4d8a-a427-9edfe53afc54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:31.582357 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78709c0f-1ff6-4d8a-a427-9edfe53afc54 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:31.595325 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 166/334] 10.222.0.40 () {56 vars in 1308 bytes} [Fri Aug 5 10:13:31 2022] GET /identity/v3/role_assignments?user.id=399007ba69804ca7ac9376d1f5e728c2&scope.system=all&role.id=c4ba94fa4bd34a09ba87d357142d9db8 => generated 510 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:32.857539 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8e45a-5d59-47a6-a441-e9ba77072661 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:32.858557 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8e45a-5d59-47a6-a441-e9ba77072661 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:32.859103 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8e45a-5d59-47a6-a441-e9ba77072661 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:32.860722 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 169/335] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:32.866230 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c26cea1-af36-418f-b315-3399d64a61fa None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:32.866671 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c26cea1-af36-418f-b315-3399d64a61fa None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:32.866848 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c26cea1-af36-418f-b315-3399d64a61fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:32.891891 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-1c26cea1-af36-418f-b315-3399d64a61fa None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:32.893084 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1c26cea1-af36-418f-b315-3399d64a61fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:32.977633 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 167/336] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:32.980457 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28a0b2-bc12-4beb-ab3f-0552543c247b None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:32.980947 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28a0b2-bc12-4beb-ab3f-0552543c247b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:32.981424 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28a0b2-bc12-4beb-ab3f-0552543c247b None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:32.987366 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 170/337] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:32.992630 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f9bad-6519-4c6c-a117-5a54de5630f7 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:32.992920 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f9bad-6519-4c6c-a117-5a54de5630f7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:32.993198 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f9bad-6519-4c6c-a117-5a54de5630f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:33.007993 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc28a0b2-bc12-4beb-ab3f-0552543c247b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:33.019972 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-4c4f9bad-6519-4c6c-a117-5a54de5630f7 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:33.021181 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4c4f9bad-6519-4c6c-a117-5a54de5630f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:33.162942 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 168/338] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:33.200398 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f9bad-6519-4c6c-a117-5a54de5630f7 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:33.211573 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc28a0b2-bc12-4beb-ab3f-0552543c247b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:33.219259 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:33.307636 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad1bb6-a0bc-482c-890a-0b12108a3283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:33.308688 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad1bb6-a0bc-482c-890a-0b12108a3283 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:33.309166 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad1bb6-a0bc-482c-890a-0b12108a3283 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:33.309654 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad1bb6-a0bc-482c-890a-0b12108a3283 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:33.316079 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:33.339884 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ad1bb6-a0bc-482c-890a-0b12108a3283 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:33.345545 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-90ad1bb6-a0bc-482c-890a-0b12108a3283 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 10:13:33.345545 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:33.345545 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:33.345545 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:33.345545 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:33.345545 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:33.345545 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:33.345545 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:33.345545 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:33.345545 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:33.345545 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:33.345545 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:33.345545 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:33.345545 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:33.345545 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:33.345545 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:33.346453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:33.346453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:33.346453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:33.346453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:33.346453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:33.346453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:33.346453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:33.346453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:33.346453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:33.346453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:33.346453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:33.346453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:33.346453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:33.346453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:33.346453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:33.346453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:33.347269 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:33.347269 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:33.347269 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:33.347269 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:33.347269 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:33.347269 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:33.347269 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:33.347269 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:33.347269 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:33.347269 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:33.347269 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 10:13:33.347269 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:33.347269 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 171/339] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 5 10:13:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:33.355071 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad1bb6-a0bc-482c-890a-0b12108a3283 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:33.372813 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f9bad-6519-4c6c-a117-5a54de5630f7 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:33.433000 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77657c2-f560-49d1-994a-997b7eb073da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:33.433585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77657c2-f560-49d1-994a-997b7eb073da None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:33.433864 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77657c2-f560-49d1-994a-997b7eb073da None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:33.434134 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77657c2-f560-49d1-994a-997b7eb073da None admin] PATH_INFO: `/v3/roles/member` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:33.458109 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77657c2-f560-49d1-994a-997b7eb073da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:33.463472 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-b77657c2-f560-49d1-994a-997b7eb073da None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 10:13:33.463472 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:33.463472 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:33.463472 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:33.463472 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:33.463472 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:33.463472 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:33.463472 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:33.463472 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:33.463472 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:33.463472 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:33.463472 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:33.463472 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:33.463472 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:33.463472 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:33.463472 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:33.464335 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:33.464335 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:33.464335 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:33.464335 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:33.464335 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:33.464335 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:33.464335 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:33.464335 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:33.464335 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:33.464335 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:33.464335 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:33.464335 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:33.464335 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:33.464335 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:33.464335 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:33.464335 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:33.465115 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:33.465115 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:33.465115 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:33.465115 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:33.465115 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:33.465115 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:33.465115 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:33.465115 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:33.465115 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:33.465115 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:33.465115 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 10:13:33.465115 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:13:33.465706 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 169/340] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:13:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:13:33.471041 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77657c2-f560-49d1-994a-997b7eb073da None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:33.572190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:33.573056 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad1bb6-a0bc-482c-890a-0b12108a3283 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:33.614425 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7c996a-1a99-4359-8ea6-924bc82db8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:33.615096 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7c996a-1a99-4359-8ea6-924bc82db8b2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:33.615380 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7c996a-1a99-4359-8ea6-924bc82db8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:33.615655 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7c996a-1a99-4359-8ea6-924bc82db8b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:33.616384 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7c996a-1a99-4359-8ea6-924bc82db8b2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:33.616692 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7c996a-1a99-4359-8ea6-924bc82db8b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:33.618094 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7c996a-1a99-4359-8ea6-924bc82db8b2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:33.632679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 172/341] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:13:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:33.641893 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7c996a-1a99-4359-8ea6-924bc82db8b2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:33.661830 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77657c2-f560-49d1-994a-997b7eb073da None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:33.726602 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:33.747617 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa3dfe-0df5-4cfa-8c8a-609088253993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:33.748268 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa3dfe-0df5-4cfa-8c8a-609088253993 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:33.748743 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa3dfe-0df5-4cfa-8c8a-609088253993 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:33.748870 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa3dfe-0df5-4cfa-8c8a-609088253993 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:33.749583 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befa3dfe-0df5-4cfa-8c8a-609088253993 None admin] RBAC: Adding query filter params (name=member) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:33.749888 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befa3dfe-0df5-4cfa-8c8a-609088253993 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:33.751322 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befa3dfe-0df5-4cfa-8c8a-609088253993 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:33.765190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 170/342] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 10:13:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:33.771056 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa3dfe-0df5-4cfa-8c8a-609088253993 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:33.784261 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7c996a-1a99-4359-8ea6-924bc82db8b2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:33.836735 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2a466e-8bec-4218-9473-b98542c20a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:33.837389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a466e-8bec-4218-9473-b98542c20a58 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:33.837676 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a466e-8bec-4218-9473-b98542c20a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:33.837958 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a466e-8bec-4218-9473-b98542c20a58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:33.938825 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2a466e-8bec-4218-9473-b98542c20a58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:33.940350 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2a466e-8bec-4218-9473-b98542c20a58 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:33.987992 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa3dfe-0df5-4cfa-8c8a-609088253993 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:34.016086 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:34.036164 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcce56e3-7e3b-4abd-9124-b3ddf5105adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:34.037381 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcce56e3-7e3b-4abd-9124-b3ddf5105adb None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:34.040502 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcce56e3-7e3b-4abd-9124-b3ddf5105adb None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:34.040979 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcce56e3-7e3b-4abd-9124-b3ddf5105adb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:34.057984 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:34.115260 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 173/343] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:13:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 10:13:34.116633 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d829d-d405-47b2-bbdf-9a56af1ddd50 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:34.116924 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d829d-d405-47b2-bbdf-9a56af1ddd50 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:34.117202 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d829d-d405-47b2-bbdf-9a56af1ddd50 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:34.118716 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 174/344] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:34.123370 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d829d-d405-47b2-bbdf-9a56af1ddd50 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:34.180500 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcce56e3-7e3b-4abd-9124-b3ddf5105adb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:34.181985 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcce56e3-7e3b-4abd-9124-b3ddf5105adb None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:34.261046 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d829d-d405-47b2-bbdf-9a56af1ddd50 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:34.307917 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59382f8a-85fb-4fa1-8c32-219e3400448a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:34.308965 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59382f8a-85fb-4fa1-8c32-219e3400448a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:34.309384 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59382f8a-85fb-4fa1-8c32-219e3400448a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:34.309711 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59382f8a-85fb-4fa1-8c32-219e3400448a None admin] PATH_INFO: `/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:34.310974 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59382f8a-85fb-4fa1-8c32-219e3400448a None admin] RBAC: Authorizing `identity:get_project(project_id=a9d326f0c0fa4a38aa7ba8c8c6797c5a)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:34.312499 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59382f8a-85fb-4fa1-8c32-219e3400448a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:34.315095 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 175/345] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:13:34 2022] GET /identity/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:34.316343 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31db1fa-5c02-4723-a978-7deaafcc2dc4 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:34.316631 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31db1fa-5c02-4723-a978-7deaafcc2dc4 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:34.316972 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31db1fa-5c02-4723-a978-7deaafcc2dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:34.326928 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 171/346] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:13:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 10:13:34.328224 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d8fd2-bd02-4334-af4b-fa7ee9d7f56b None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:34.328551 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d8fd2-bd02-4334-af4b-fa7ee9d7f56b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:34.328825 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d8fd2-bd02-4334-af4b-fa7ee9d7f56b None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:34.330347 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 172/347] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:34.331062 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d8fd2-bd02-4334-af4b-fa7ee9d7f56b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:34.343276 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c31db1fa-5c02-4723-a978-7deaafcc2dc4 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:34.344492 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c31db1fa-5c02-4723-a978-7deaafcc2dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:34.371501 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:34.373988 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:34.442977 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 176/348] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:34.444580 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a16c09-4695-4b35-8f16-329c83aaf591 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:34.446649 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a16c09-4695-4b35-8f16-329c83aaf591 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:34.447313 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a16c09-4695-4b35-8f16-329c83aaf591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:34.476164 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f5a16c09-4695-4b35-8f16-329c83aaf591 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:34.477633 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f5a16c09-4695-4b35-8f16-329c83aaf591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:34.485374 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d8fd2-bd02-4334-af4b-fa7ee9d7f56b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:34.524155 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aaff26-b533-4820-8365-e9081ec693ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:34.524732 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aaff26-b533-4820-8365-e9081ec693ff None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:34.525013 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aaff26-b533-4820-8365-e9081ec693ff None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:34.525339 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aaff26-b533-4820-8365-e9081ec693ff None admin] PATH_INFO: `/v3/groups/bded54c235c74f12a167629af5a6e94e` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:34.533091 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aaff26-b533-4820-8365-e9081ec693ff None admin] RBAC: Authorizing `identity:get_group(group_id=bded54c235c74f12a167629af5a6e94e)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:34.534885 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aaff26-b533-4820-8365-e9081ec693ff None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:34.536943 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 173/349] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 5 10:13:34 2022] GET /identity/v3/groups/bded54c235c74f12a167629af5a6e94e => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:34.538058 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aaff26-b533-4820-8365-e9081ec693ff None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:34.569321 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 177/350] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:34.570506 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a16c09-4695-4b35-8f16-329c83aaf591 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:34.687339 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:34.691906 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aaff26-b533-4820-8365-e9081ec693ff None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:34.696622 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:34.713539 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a16c09-4695-4b35-8f16-329c83aaf591 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:34.735602 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb9ec86-0a35-4400-9b65-dabb3cba93a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:34.735602 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9ec86-0a35-4400-9b65-dabb3cba93a1 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:34.735602 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9ec86-0a35-4400-9b65-dabb3cba93a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:34.735602 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9ec86-0a35-4400-9b65-dabb3cba93a1 None admin] PATH_INFO: `/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:34.736008 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb9ec86-0a35-4400-9b65-dabb3cba93a1 None admin] RBAC: Authorizing `identity:get_project(project_id=a9d326f0c0fa4a38aa7ba8c8c6797c5a)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:34.737821 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb9ec86-0a35-4400-9b65-dabb3cba93a1 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:34.742058 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 174/351] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:13:34 2022] GET /identity/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:34.742058 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb9ec86-0a35-4400-9b65-dabb3cba93a1 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:34.758799 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096edb93-506a-483d-9b70-9f21ff7acb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:34.759501 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096edb93-506a-483d-9b70-9f21ff7acb99 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:34.759781 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096edb93-506a-483d-9b70-9f21ff7acb99 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:34.760055 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096edb93-506a-483d-9b70-9f21ff7acb99 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:34.772597 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096edb93-506a-483d-9b70-9f21ff7acb99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:34.778125 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-096edb93-506a-483d-9b70-9f21ff7acb99 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 10:13:34.778125 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:34.778125 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:34.778125 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:34.778125 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:34.778125 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:34.778125 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:34.778125 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:34.778125 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:34.778125 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:34.778125 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:34.778125 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:34.778125 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:34.778125 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:34.778125 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:34.778125 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:34.779039 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:34.779039 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:34.779039 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:34.779039 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:34.779039 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:34.779039 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:34.779039 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:34.779039 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:34.779039 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:34.779039 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:34.779039 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:34.779039 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:34.779039 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:34.779039 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:34.779039 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:34.779039 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:34.779991 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:34.779991 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:34.779991 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:34.779991 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:34.779991 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:34.779991 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:34.779991 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:34.779991 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:34.779991 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:34.779991 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:34.779991 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 10:13:34.779991 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:34.779991 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 178/352] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:13:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:34.780648 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096edb93-506a-483d-9b70-9f21ff7acb99 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:34.869493 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb9ec86-0a35-4400-9b65-dabb3cba93a1 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:34.905009 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096edb93-506a-483d-9b70-9f21ff7acb99 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:34.910890 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed9e0a-31cb-47fb-909f-d242ce63df88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:34.911652 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed9e0a-31cb-47fb-909f-d242ce63df88 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:34.912005 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed9e0a-31cb-47fb-909f-d242ce63df88 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:34.912347 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed9e0a-31cb-47fb-909f-d242ce63df88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:34.913267 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aed9e0a-31cb-47fb-909f-d242ce63df88 None admin] RBAC: Adding query filter params (group.id=bded54c235c74f12a167629af5a6e94e, role.id=57a070b43ad940f1886c69721dd304c2, scope.project.id=a9d326f0c0fa4a38aa7ba8c8c6797c5a) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:34.913583 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aed9e0a-31cb-47fb-909f-d242ce63df88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:34.915112 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aed9e0a-31cb-47fb-909f-d242ce63df88 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:34.922602 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 175/353] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Aug 5 10:13:34 2022] GET /identity/v3/role_assignments?group.id=bded54c235c74f12a167629af5a6e94e&scope.project.id=a9d326f0c0fa4a38aa7ba8c8c6797c5a&role.id=57a070b43ad940f1886c69721dd304c2 => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:34.923871 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9bd727-05ea-44a3-800c-2dbb9f3e7a69 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:34.924161 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9bd727-05ea-44a3-800c-2dbb9f3e7a69 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:34.924505 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9bd727-05ea-44a3-800c-2dbb9f3e7a69 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:34.925726 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 176/354] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:34.926418 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9bd727-05ea-44a3-800c-2dbb9f3e7a69 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:34.947844 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9d694-4e69-49ef-908f-e8d69f4adf61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:34.949255 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9d694-4e69-49ef-908f-e8d69f4adf61 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:34.949733 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9d694-4e69-49ef-908f-e8d69f4adf61 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:34.950221 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9d694-4e69-49ef-908f-e8d69f4adf61 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:34.967621 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b9d694-4e69-49ef-908f-e8d69f4adf61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:34.973251 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-50b9d694-4e69-49ef-908f-e8d69f4adf61 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 10:13:34.973251 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:34.973251 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:34.973251 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:34.973251 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:34.973251 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:34.973251 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:34.973251 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:34.973251 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:34.973251 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:34.973251 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:34.973251 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:34.973251 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:34.973251 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:34.973251 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:34.973251 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:34.974197 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:34.974197 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:34.974197 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:34.974197 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:34.974197 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:34.974197 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:34.974197 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:34.974197 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:34.974197 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:34.974197 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:34.974197 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:34.974197 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:34.974197 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:34.974197 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:34.974197 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:34.974197 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:34.976454 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:34.976454 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:34.976454 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:34.976454 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:34.976454 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:34.976454 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:34.976454 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:34.976454 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:34.976454 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:34.976454 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:34.976454 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 10:13:34.976454 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:34.977057 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:34.978675 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 179/355] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:13:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:34.979811 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9d694-4e69-49ef-908f-e8d69f4adf61 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:34.992598 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:35.098435 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9bd727-05ea-44a3-800c-2dbb9f3e7a69 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:35.116656 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9d694-4e69-49ef-908f-e8d69f4adf61 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:35.140493 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff640ae-dc3a-4c54-a4c5-16fe82f08fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:35.141610 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff640ae-dc3a-4c54-a4c5-16fe82f08fb6 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:35.141895 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff640ae-dc3a-4c54-a4c5-16fe82f08fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:35.142164 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff640ae-dc3a-4c54-a4c5-16fe82f08fb6 None admin] PATH_INFO: `/v3/groups/bded54c235c74f12a167629af5a6e94e` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:35.143620 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff640ae-dc3a-4c54-a4c5-16fe82f08fb6 None admin] RBAC: Authorizing `identity:get_group(group_id=bded54c235c74f12a167629af5a6e94e)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:35.145240 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff640ae-dc3a-4c54-a4c5-16fe82f08fb6 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:35.147644 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 177/356] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 5 10:13:34 2022] GET /identity/v3/groups/bded54c235c74f12a167629af5a6e94e => generated 226 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:35.151027 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110dd888-7490-489b-bc72-80bdae072a7b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:35.151027 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110dd888-7490-489b-bc72-80bdae072a7b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:35.151027 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110dd888-7490-489b-bc72-80bdae072a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:35.170555 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a62d8a-943b-41ba-9c0e-6add0829ea16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:35.171673 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-110dd888-7490-489b-bc72-80bdae072a7b None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:35.172026 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a62d8a-943b-41ba-9c0e-6add0829ea16 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:35.172498 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a62d8a-943b-41ba-9c0e-6add0829ea16 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:35.172864 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-110dd888-7490-489b-bc72-80bdae072a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:35.173238 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a62d8a-943b-41ba-9c0e-6add0829ea16 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:35.174149 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a62d8a-943b-41ba-9c0e-6add0829ea16 None admin] RBAC: Adding query filter params (name=member) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:35.174460 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a62d8a-943b-41ba-9c0e-6add0829ea16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:35.175906 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a62d8a-943b-41ba-9c0e-6add0829ea16 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:35.188403 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 180/357] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 10:13:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:35.189334 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a62d8a-943b-41ba-9c0e-6add0829ea16 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:35.260446 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 178/358] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:35.261326 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110dd888-7490-489b-bc72-80bdae072a7b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:35.271883 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:35.290977 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:35.317854 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a62d8a-943b-41ba-9c0e-6add0829ea16 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:35.355306 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a7b26b-a74c-4349-97b2-982b078ce125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:35.355859 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7b26b-a74c-4349-97b2-982b078ce125 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:35.356143 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7b26b-a74c-4349-97b2-982b078ce125 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:35.356419 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7b26b-a74c-4349-97b2-982b078ce125 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:35.357134 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a7b26b-a74c-4349-97b2-982b078ce125 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:35.357440 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a7b26b-a74c-4349-97b2-982b078ce125 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:35.359378 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a7b26b-a74c-4349-97b2-982b078ce125 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:35.368687 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 181/359] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 10:13:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:35.371415 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df7fa5c-611a-4f59-afe7-d46637e4ded7 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:35.371536 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df7fa5c-611a-4f59-afe7-d46637e4ded7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:35.371779 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df7fa5c-611a-4f59-afe7-d46637e4ded7 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:35.378077 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 182/360] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:35.379920 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df7fa5c-611a-4f59-afe7-d46637e4ded7 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:35.396023 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110dd888-7490-489b-bc72-80bdae072a7b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:35.437005 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec463df-5c7c-46df-b241-126c08dbd703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:35.437784 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec463df-5c7c-46df-b241-126c08dbd703 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:35.438058 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec463df-5c7c-46df-b241-126c08dbd703 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:35.438716 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec463df-5c7c-46df-b241-126c08dbd703 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:35.439826 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec463df-5c7c-46df-b241-126c08dbd703 None admin] RBAC: Adding query filter params (group.id=bded54c235c74f12a167629af5a6e94e, role.id=c4ba94fa4bd34a09ba87d357142d9db8, scope.project.id=a9d326f0c0fa4a38aa7ba8c8c6797c5a) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:35.440134 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec463df-5c7c-46df-b241-126c08dbd703 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:35.441721 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec463df-5c7c-46df-b241-126c08dbd703 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:35.449577 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 179/361] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Aug 5 10:13:35 2022] GET /identity/v3/role_assignments?group.id=bded54c235c74f12a167629af5a6e94e&scope.project.id=a9d326f0c0fa4a38aa7ba8c8c6797c5a&role.id=c4ba94fa4bd34a09ba87d357142d9db8 => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:35.450199 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec463df-5c7c-46df-b241-126c08dbd703 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:35.506283 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df7fa5c-611a-4f59-afe7-d46637e4ded7 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:35.542211 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:35.549679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f4c1b-f84d-4398-af22-c0d00da34081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:35.550324 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f4c1b-f84d-4398-af22-c0d00da34081 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:35.550708 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f4c1b-f84d-4398-af22-c0d00da34081 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:35.551036 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f4c1b-f84d-4398-af22-c0d00da34081 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:35.554494 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:35.574430 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec463df-5c7c-46df-b241-126c08dbd703 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:35.613789 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684065a2-e9f0-49c0-bdcb-5e5bd35c2930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:35.614496 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684065a2-e9f0-49c0-bdcb-5e5bd35c2930 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:35.614789 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684065a2-e9f0-49c0-bdcb-5e5bd35c2930 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:35.615085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684065a2-e9f0-49c0-bdcb-5e5bd35c2930 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:35.628484 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684065a2-e9f0-49c0-bdcb-5e5bd35c2930 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:35.630635 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63f4c1b-f84d-4398-af22-c0d00da34081 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:35.632201 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63f4c1b-f84d-4398-af22-c0d00da34081 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:35.634224 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-684065a2-e9f0-49c0-bdcb-5e5bd35c2930 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 10:13:35.634224 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:35.634224 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:35.634224 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:35.634224 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:35.634224 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:35.634224 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:35.634224 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:35.634224 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:35.634224 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:35.634224 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:35.634224 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:35.634224 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:35.634224 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:35.634224 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:35.634224 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:35.635168 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:35.635168 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:35.635168 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:35.635168 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:35.635168 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:35.635168 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:35.635168 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:35.635168 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:35.635168 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:35.635168 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:35.635168 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:35.635168 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:35.635168 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:35.635168 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:35.635168 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:35.635168 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:35.636415 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:35.636415 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:35.636415 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:35.636415 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:35.636415 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:35.636415 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:35.636415 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:35.636415 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:35.636415 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:35.636415 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:35.636415 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 10:13:35.636415 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:13:35.636415 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 180/362] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 5 10:13:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:35.636415 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684065a2-e9f0-49c0-bdcb-5e5bd35c2930 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:35.748364 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 183/363] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:13:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 10:13:35.749676 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4de1d-c624-4b09-a56b-d911b78886e6 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:35.749987 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4de1d-c624-4b09-a56b-d911b78886e6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:35.750175 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684065a2-e9f0-49c0-bdcb-5e5bd35c2930 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:35.750271 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4de1d-c624-4b09-a56b-d911b78886e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:35.773246 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-99b4de1d-c624-4b09-a56b-d911b78886e6 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:35.775266 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-99b4de1d-c624-4b09-a56b-d911b78886e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:35.789977 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3764463-fb9e-4529-8fa5-23ae3b3fef13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:35.790689 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3764463-fb9e-4529-8fa5-23ae3b3fef13 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:35.791006 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3764463-fb9e-4529-8fa5-23ae3b3fef13 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:35.791280 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3764463-fb9e-4529-8fa5-23ae3b3fef13 None admin] PATH_INFO: `/v3/users/79c11a00e4994d83919c7f59026b4a12` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:35.792449 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3764463-fb9e-4529-8fa5-23ae3b3fef13 None admin] RBAC: Authorizing `identity:get_user(user_id=79c11a00e4994d83919c7f59026b4a12)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:35.793907 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3764463-fb9e-4529-8fa5-23ae3b3fef13 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:35.795936 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 181/364] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:13:35 2022] GET /identity/v3/users/79c11a00e4994d83919c7f59026b4a12 => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:35.796779 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3764463-fb9e-4529-8fa5-23ae3b3fef13 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:35.807786 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:35.813297 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:35.881954 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 184/365] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:35.883560 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b4de1d-c624-4b09-a56b-d911b78886e6 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:35.921839 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3764463-fb9e-4529-8fa5-23ae3b3fef13 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:35.961351 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5145a5a9-033d-4022-aa60-ce9c02d9955a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:35.962026 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5145a5a9-033d-4022-aa60-ce9c02d9955a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:35.962308 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5145a5a9-033d-4022-aa60-ce9c02d9955a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:35.962580 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5145a5a9-033d-4022-aa60-ce9c02d9955a None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:35.963353 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5145a5a9-033d-4022-aa60-ce9c02d9955a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:35.963654 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5145a5a9-033d-4022-aa60-ce9c02d9955a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:35.965024 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5145a5a9-033d-4022-aa60-ce9c02d9955a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:35.973990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 182/366] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:13:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:35.974877 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5145a5a9-033d-4022-aa60-ce9c02d9955a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:36.019045 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b4de1d-c624-4b09-a56b-d911b78886e6 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:36.056845 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a1766-4d0a-49f3-9997-1c03e54a5621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:36.057866 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a1766-4d0a-49f3-9997-1c03e54a5621 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:36.058334 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:36.058487 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a1766-4d0a-49f3-9997-1c03e54a5621 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:36.058806 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a1766-4d0a-49f3-9997-1c03e54a5621 None admin] PATH_INFO: `/v3/projects/400a983b4cd947d487b7fa86a5f5c339` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:36.061372 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1a1766-4d0a-49f3-9997-1c03e54a5621 None admin] RBAC: Authorizing `identity:get_project(project_id=400a983b4cd947d487b7fa86a5f5c339)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:36.062843 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1a1766-4d0a-49f3-9997-1c03e54a5621 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:36.065217 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 185/367] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:13:35 2022] GET /identity/v3/projects/400a983b4cd947d487b7fa86a5f5c339 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:36.066069 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a1766-4d0a-49f3-9997-1c03e54a5621 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:36.088280 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5145a5a9-033d-4022-aa60-ce9c02d9955a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:36.100520 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:36.123877 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeb870b-5c07-4adc-b050-87621f069705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:36.125158 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb870b-5c07-4adc-b050-87621f069705 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:36.125576 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb870b-5c07-4adc-b050-87621f069705 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:36.126006 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb870b-5c07-4adc-b050-87621f069705 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:36.126949 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeb870b-5c07-4adc-b050-87621f069705 None admin] RBAC: Adding query filter params (role.id=cf5de414a035475b8b75626cb19a2de2, user.id=79c11a00e4994d83919c7f59026b4a12, scope.system=all) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:36.127392 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeb870b-5c07-4adc-b050-87621f069705 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:36.129075 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeb870b-5c07-4adc-b050-87621f069705 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:36.142162 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 183/368] 10.222.0.40 () {56 vars in 1308 bytes} [Fri Aug 5 10:13:35 2022] GET /identity/v3/role_assignments?user.id=79c11a00e4994d83919c7f59026b4a12&scope.system=all&role.id=cf5de414a035475b8b75626cb19a2de2 => generated 510 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:36.143317 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeb870b-5c07-4adc-b050-87621f069705 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:36.222549 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a1766-4d0a-49f3-9997-1c03e54a5621 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:36.300785 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a539376-e599-4558-8a1f-2b2b6d8e9576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:36.301994 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a539376-e599-4558-8a1f-2b2b6d8e9576 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:36.303705 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a539376-e599-4558-8a1f-2b2b6d8e9576 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:36.304266 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a539376-e599-4558-8a1f-2b2b6d8e9576 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:36.324429 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeb870b-5c07-4adc-b050-87621f069705 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:36.327414 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a539376-e599-4558-8a1f-2b2b6d8e9576 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:36.333391 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-1a539376-e599-4558-8a1f-2b2b6d8e9576 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:13:36.333391 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:36.333391 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:36.333391 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:36.333391 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:36.333391 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:36.333391 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:36.333391 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:36.333391 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:36.333391 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:36.333391 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:36.333391 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:36.333391 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:36.333391 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:36.333391 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:36.333391 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:36.334326 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:36.334326 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:36.334326 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:36.334326 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:36.334326 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:36.334326 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:36.334326 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:36.334326 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:36.334326 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:36.334326 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:36.334326 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:36.334326 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:36.334326 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:36.334326 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:36.334326 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:36.334326 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:36.335163 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:36.335163 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:36.335163 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:36.335163 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:36.335163 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:36.335163 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:36.335163 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:36.335163 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:36.335163 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:36.335163 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:36.335163 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:13:36.335163 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:36.345189 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 186/369] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 10:13:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:36.346450 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a539376-e599-4558-8a1f-2b2b6d8e9576 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:36.388487 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:36.401813 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa0cf5a-ea40-4f1a-95ba-25ee6f021945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:36.402381 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa0cf5a-ea40-4f1a-95ba-25ee6f021945 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:36.402981 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa0cf5a-ea40-4f1a-95ba-25ee6f021945 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:36.403269 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa0cf5a-ea40-4f1a-95ba-25ee6f021945 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:36.489585 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:36.508972 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a539376-e599-4558-8a1f-2b2b6d8e9576 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:36.513881 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa0cf5a-ea40-4f1a-95ba-25ee6f021945 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:36.515594 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa0cf5a-ea40-4f1a-95ba-25ee6f021945 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:36.565645 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8204db-0fb0-4ea8-9a9a-b53161bd8d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:36.566311 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8204db-0fb0-4ea8-9a9a-b53161bd8d81 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:36.566591 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8204db-0fb0-4ea8-9a9a-b53161bd8d81 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:36.566865 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8204db-0fb0-4ea8-9a9a-b53161bd8d81 None admin] PATH_INFO: `/v3/groups/bded54c235c74f12a167629af5a6e94e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:36.569693 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8204db-0fb0-4ea8-9a9a-b53161bd8d81 None admin] RBAC: Authorizing `identity:get_group(group_id=bded54c235c74f12a167629af5a6e94e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:36.571196 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8204db-0fb0-4ea8-9a9a-b53161bd8d81 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:36.573114 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 187/370] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 5 10:13:36 2022] GET /identity/v3/groups/bded54c235c74f12a167629af5a6e94e => generated 226 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:36.573878 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8204db-0fb0-4ea8-9a9a-b53161bd8d81 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:36.650040 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 184/371] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:13:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 10:13:36.650944 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa0cf5a-ea40-4f1a-95ba-25ee6f021945 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:36.695579 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:36.700346 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8204db-0fb0-4ea8-9a9a-b53161bd8d81 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:36.745061 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eaadc6-fe53-4a22-99c4-dc7c6ba37a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:36.745888 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eaadc6-fe53-4a22-99c4-dc7c6ba37a05 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:36.746235 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eaadc6-fe53-4a22-99c4-dc7c6ba37a05 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:36.746572 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eaadc6-fe53-4a22-99c4-dc7c6ba37a05 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:36.747490 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eaadc6-fe53-4a22-99c4-dc7c6ba37a05 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:36.747859 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eaadc6-fe53-4a22-99c4-dc7c6ba37a05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:36.749388 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eaadc6-fe53-4a22-99c4-dc7c6ba37a05 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:36.759053 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 188/372] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:13:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:36.759942 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eaadc6-fe53-4a22-99c4-dc7c6ba37a05 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:36.780588 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa0cf5a-ea40-4f1a-95ba-25ee6f021945 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:36.780780 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:36.837950 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5fbc88-e821-4ca8-95c0-7129f2e685b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:36.838733 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5fbc88-e821-4ca8-95c0-7129f2e685b0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:36.839037 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5fbc88-e821-4ca8-95c0-7129f2e685b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:36.839626 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5fbc88-e821-4ca8-95c0-7129f2e685b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:36.842596 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5fbc88-e821-4ca8-95c0-7129f2e685b0 None admin] RBAC: Adding query filter params (group.id=bded54c235c74f12a167629af5a6e94e, role.id=c4ba94fa4bd34a09ba87d357142d9db8, scope.project.id=400a983b4cd947d487b7fa86a5f5c339) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:36.842927 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5fbc88-e821-4ca8-95c0-7129f2e685b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:36.845230 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5fbc88-e821-4ca8-95c0-7129f2e685b0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:36.852911 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 185/373] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Aug 5 10:13:36 2022] GET /identity/v3/role_assignments?group.id=bded54c235c74f12a167629af5a6e94e&scope.project.id=400a983b4cd947d487b7fa86a5f5c339&role.id=c4ba94fa4bd34a09ba87d357142d9db8 => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:36.854856 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5fbc88-e821-4ca8-95c0-7129f2e685b0 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:36.899536 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eaadc6-fe53-4a22-99c4-dc7c6ba37a05 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:36.947632 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35aad92-9568-4037-85b0-715040684456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:36.948215 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35aad92-9568-4037-85b0-715040684456 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:36.948496 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35aad92-9568-4037-85b0-715040684456 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:36.948769 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35aad92-9568-4037-85b0-715040684456 None admin] PATH_INFO: `/v3/projects/400a983b4cd947d487b7fa86a5f5c339` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:36.951255 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35aad92-9568-4037-85b0-715040684456 None admin] RBAC: Authorizing `identity:get_project(project_id=400a983b4cd947d487b7fa86a5f5c339)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:36.953803 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35aad92-9568-4037-85b0-715040684456 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:36.956865 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 189/374] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:13:36 2022] GET /identity/v3/projects/400a983b4cd947d487b7fa86a5f5c339 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:36.964041 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35aad92-9568-4037-85b0-715040684456 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:36.987330 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:36.995550 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5fbc88-e821-4ca8-95c0-7129f2e685b0 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:37.060489 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6c54f6-a8c3-4b01-b402-b2aa1c2034cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:37.061226 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c54f6-a8c3-4b01-b402-b2aa1c2034cb None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:37.061504 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c54f6-a8c3-4b01-b402-b2aa1c2034cb None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:37.061778 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c54f6-a8c3-4b01-b402-b2aa1c2034cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:37.128452 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:37.149746 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35aad92-9568-4037-85b0-715040684456 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:37.181624 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6c54f6-a8c3-4b01-b402-b2aa1c2034cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:37.183171 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6c54f6-a8c3-4b01-b402-b2aa1c2034cb None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:37.247633 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa23131-aa91-43fc-8039-043c0b277711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:37.248797 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa23131-aa91-43fc-8039-043c0b277711 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:37.249180 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa23131-aa91-43fc-8039-043c0b277711 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:37.249553 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa23131-aa91-43fc-8039-043c0b277711 None admin] PATH_INFO: `/v3/groups/bded54c235c74f12a167629af5a6e94e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:37.250878 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa23131-aa91-43fc-8039-043c0b277711 None admin] RBAC: Authorizing `identity:get_group(group_id=bded54c235c74f12a167629af5a6e94e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:37.252519 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa23131-aa91-43fc-8039-043c0b277711 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:37.254548 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 190/375] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 5 10:13:36 2022] GET /identity/v3/groups/bded54c235c74f12a167629af5a6e94e => generated 226 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:37.261066 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa23131-aa91-43fc-8039-043c0b277711 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:37.292100 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:37.296059 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 186/376] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:13:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 10:13:37.304165 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6c54f6-a8c3-4b01-b402-b2aa1c2034cb None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:37.414129 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6c54f6-a8c3-4b01-b402-b2aa1c2034cb None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:37.451896 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d3f2a-e855-4a33-ae9a-d3f04d4df107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:37.452553 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d3f2a-e855-4a33-ae9a-d3f04d4df107 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:37.452829 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d3f2a-e855-4a33-ae9a-d3f04d4df107 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:37.453097 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d3f2a-e855-4a33-ae9a-d3f04d4df107 None admin] PATH_INFO: `/v3/projects/be89c43472bc49a2bd0e46cddf175958` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:37.454320 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294d3f2a-e855-4a33-ae9a-d3f04d4df107 None admin] RBAC: Authorizing `identity:get_project(project_id=be89c43472bc49a2bd0e46cddf175958)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:37.455811 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294d3f2a-e855-4a33-ae9a-d3f04d4df107 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:37.458785 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 187/377] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:13:37 2022] GET /identity/v3/projects/be89c43472bc49a2bd0e46cddf175958 => generated 344 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:13:37.463384 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d3f2a-e855-4a33-ae9a-d3f04d4df107 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:37.486928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa23131-aa91-43fc-8039-043c0b277711 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:37.524827 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17fa60f-2778-4659-a37d-cea260ea468e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:37.525806 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17fa60f-2778-4659-a37d-cea260ea468e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:37.526299 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17fa60f-2778-4659-a37d-cea260ea468e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:37.526767 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17fa60f-2778-4659-a37d-cea260ea468e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:37.527872 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17fa60f-2778-4659-a37d-cea260ea468e None admin] RBAC: Adding query filter params (group.id=bded54c235c74f12a167629af5a6e94e, role.id=57a070b43ad940f1886c69721dd304c2, scope.project.id=400a983b4cd947d487b7fa86a5f5c339) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:37.528503 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17fa60f-2778-4659-a37d-cea260ea468e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:37.530180 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17fa60f-2778-4659-a37d-cea260ea468e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:37.534448 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:37.538049 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 191/378] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Aug 5 10:13:37 2022] GET /identity/v3/role_assignments?group.id=bded54c235c74f12a167629af5a6e94e&scope.project.id=400a983b4cd947d487b7fa86a5f5c339&role.id=57a070b43ad940f1886c69721dd304c2 => generated 260 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:13:37.539532 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb71b8b-4852-4d82-89fa-c5cf21c84236 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:37.540015 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb71b8b-4852-4d82-89fa-c5cf21c84236 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:37.540505 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb71b8b-4852-4d82-89fa-c5cf21c84236 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:37.547189 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 192/379] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:37.549274 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da80685-dc98-4326-9862-9837afc1c335 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:37.549773 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da80685-dc98-4326-9862-9837afc1c335 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:37.550247 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da80685-dc98-4326-9862-9837afc1c335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:37.570459 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-7da80685-dc98-4326-9862-9837afc1c335 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:37.572080 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7da80685-dc98-4326-9862-9837afc1c335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:37.588092 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:37.595399 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d3f2a-e855-4a33-ae9a-d3f04d4df107 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:37.643498 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9995aec-8615-4e56-8828-02f99abe876d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:37.644170 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9995aec-8615-4e56-8828-02f99abe876d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:37.644492 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9995aec-8615-4e56-8828-02f99abe876d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:37.644759 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9995aec-8615-4e56-8828-02f99abe876d None admin] PATH_INFO: `/v3/groups/f8ba520a72c14d3f9de2539d0c047a2c` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:37.657942 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9995aec-8615-4e56-8828-02f99abe876d None admin] RBAC: Authorizing `identity:get_group(group_id=f8ba520a72c14d3f9de2539d0c047a2c)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:37.659462 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9995aec-8615-4e56-8828-02f99abe876d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:37.661435 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 188/380] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 5 10:13:37 2022] GET /identity/v3/groups/f8ba520a72c14d3f9de2539d0c047a2c => generated 229 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:37.667191 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9995aec-8615-4e56-8828-02f99abe876d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:37.693104 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 193/381] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 10:13:37.736495 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da80685-dc98-4326-9862-9837afc1c335 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:37.841102 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9995aec-8615-4e56-8828-02f99abe876d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:37.871029 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:37.883323 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffee560a-0b5e-411c-97b1-f1eaa93872ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:37.883954 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee560a-0b5e-411c-97b1-f1eaa93872ed None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:37.884229 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee560a-0b5e-411c-97b1-f1eaa93872ed None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:37.884497 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee560a-0b5e-411c-97b1-f1eaa93872ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:37.885286 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffee560a-0b5e-411c-97b1-f1eaa93872ed None admin] RBAC: Adding query filter params (group.id=f8ba520a72c14d3f9de2539d0c047a2c, role.id=daef89adcaeb48a382a0bcb03340ccbb, scope.project.id=be89c43472bc49a2bd0e46cddf175958) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:37.885593 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffee560a-0b5e-411c-97b1-f1eaa93872ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:37.887087 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffee560a-0b5e-411c-97b1-f1eaa93872ed None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:37.892512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da80685-dc98-4326-9862-9837afc1c335 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:37.893732 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 189/382] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Aug 5 10:13:37 2022] GET /identity/v3/role_assignments?group.id=f8ba520a72c14d3f9de2539d0c047a2c&scope.project.id=be89c43472bc49a2bd0e46cddf175958&role.id=daef89adcaeb48a382a0bcb03340ccbb => generated 260 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:37.933125 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b199176-1c7f-4a22-a786-e730353b9fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:37.933789 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b199176-1c7f-4a22-a786-e730353b9fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:37.934069 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b199176-1c7f-4a22-a786-e730353b9fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:37.934343 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b199176-1c7f-4a22-a786-e730353b9fa7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:37.946526 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b199176-1c7f-4a22-a786-e730353b9fa7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:37.956290 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-7b199176-1c7f-4a22-a786-e730353b9fa7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 10:13:37.956290 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:37.956290 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:37.956290 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:37.956290 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:37.956290 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:37.956290 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:37.956290 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:37.956290 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:37.956290 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:37.956290 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:37.956290 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:37.956290 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:37.956290 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:37.956290 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:37.956290 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:37.957235 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:37.957235 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:37.957235 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:37.957235 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:37.957235 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:37.957235 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:37.957235 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:37.957235 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:37.957235 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:37.957235 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:37.957235 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:37.957235 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:37.957235 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:37.957235 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:37.957235 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:37.957235 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:37.958022 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:37.958022 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:37.958022 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:37.958022 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:37.958022 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:37.958022 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:37.958022 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:37.958022 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:37.958022 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:37.958022 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:37.958022 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 10:13:37.958022 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:37.958913 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 194/383] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 5 10:13:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:37.964992 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffee560a-0b5e-411c-97b1-f1eaa93872ed None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:38.108267 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffee560a-0b5e-411c-97b1-f1eaa93872ed None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:38.149543 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e9a4a7-2f23-4baa-a392-9f6ca8c21c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:38.150199 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9a4a7-2f23-4baa-a392-9f6ca8c21c5e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:38.150480 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9a4a7-2f23-4baa-a392-9f6ca8c21c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:38.150751 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9a4a7-2f23-4baa-a392-9f6ca8c21c5e None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:38.154661 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e9a4a7-2f23-4baa-a392-9f6ca8c21c5e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:38.154661 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e9a4a7-2f23-4baa-a392-9f6ca8c21c5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:38.154661 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e9a4a7-2f23-4baa-a392-9f6ca8c21c5e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:38.163455 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 190/384] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:13:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:38.174255 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b199176-1c7f-4a22-a786-e730353b9fa7 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:38.188541 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:38.191616 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f87d5-4c6c-426d-9337-9a3c371ae6f9 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:38.192430 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f87d5-4c6c-426d-9337-9a3c371ae6f9 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:38.195144 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f87d5-4c6c-426d-9337-9a3c371ae6f9 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:38.195144 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 191/385] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 10:13:38.201740 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7390a1c-734a-47f3-a77c-4005fe24325b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:38.202507 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7390a1c-734a-47f3-a77c-4005fe24325b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:38.203069 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7390a1c-734a-47f3-a77c-4005fe24325b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:38.242193 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c7390a1c-734a-47f3-a77c-4005fe24325b None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:38.243924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c7390a1c-734a-47f3-a77c-4005fe24325b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:38.295008 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b199176-1c7f-4a22-a786-e730353b9fa7 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:38.332894 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61098e78-94d8-4d8d-9357-bc8b4acf217b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:38.333587 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61098e78-94d8-4d8d-9357-bc8b4acf217b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:38.333868 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61098e78-94d8-4d8d-9357-bc8b4acf217b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:38.334144 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61098e78-94d8-4d8d-9357-bc8b4acf217b None admin] PATH_INFO: `/v3/groups/bded54c235c74f12a167629af5a6e94e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:38.335343 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61098e78-94d8-4d8d-9357-bc8b4acf217b None admin] RBAC: Authorizing `identity:get_group(group_id=bded54c235c74f12a167629af5a6e94e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:38.336809 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61098e78-94d8-4d8d-9357-bc8b4acf217b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:38.341620 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:38.344384 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 195/386] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 5 10:13:38 2022] GET /identity/v3/groups/bded54c235c74f12a167629af5a6e94e => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:38.345648 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61098e78-94d8-4d8d-9357-bc8b4acf217b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:38.374504 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 192/387] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 10:13:38.414651 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7390a1c-734a-47f3-a77c-4005fe24325b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:38.465373 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:38.525065 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61098e78-94d8-4d8d-9357-bc8b4acf217b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:38.566665 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7390a1c-734a-47f3-a77c-4005fe24325b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:38.604133 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a70146-2e8c-4130-8938-b1c39a6bdf89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:38.605168 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a70146-2e8c-4130-8938-b1c39a6bdf89 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:38.605635 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a70146-2e8c-4130-8938-b1c39a6bdf89 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:38.606092 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a70146-2e8c-4130-8938-b1c39a6bdf89 None admin] PATH_INFO: `/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:38.608953 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de50296c-32d7-444e-97b4-59c828414204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:38.609611 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50296c-32d7-444e-97b4-59c828414204 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:38.609897 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50296c-32d7-444e-97b4-59c828414204 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:38.610172 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50296c-32d7-444e-97b4-59c828414204 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:38.610581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a70146-2e8c-4130-8938-b1c39a6bdf89 None admin] RBAC: Authorizing `identity:get_project(project_id=a9d326f0c0fa4a38aa7ba8c8c6797c5a)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:38.612277 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a70146-2e8c-4130-8938-b1c39a6bdf89 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:38.614222 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 196/388] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:13:38 2022] GET /identity/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a => generated 298 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:38.621544 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a70146-2e8c-4130-8938-b1c39a6bdf89 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:38.630602 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de50296c-32d7-444e-97b4-59c828414204 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:38.636176 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-de50296c-32d7-444e-97b4-59c828414204 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 10:13:38.636176 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:38.636176 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:38.636176 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:38.636176 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:38.636176 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:38.636176 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:38.636176 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:38.636176 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:38.636176 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:38.636176 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:38.636176 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:38.636176 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:38.636176 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:38.636176 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:38.636176 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:38.637044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:38.637044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:38.637044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:38.637044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:38.637044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:38.637044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:38.637044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:38.637044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:38.637044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:38.637044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:38.637044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:38.637044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:38.637044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:38.637044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:38.637044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:38.637044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:38.637841 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:38.637841 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:38.637841 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:38.637841 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:38.637841 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:38.637841 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:38.637841 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:38.637841 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:38.637841 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:38.637841 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:38.637841 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 10:13:38.637841 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:13:38.637841 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 193/389] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:13:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:13:38.645407 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de50296c-32d7-444e-97b4-59c828414204 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:38.716227 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:38.786885 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a70146-2e8c-4130-8938-b1c39a6bdf89 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:38.823401 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea2b83-7f2a-4544-aa1c-59895dc0b3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:38.824114 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea2b83-7f2a-4544-aa1c-59895dc0b3f3 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:38.824390 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea2b83-7f2a-4544-aa1c-59895dc0b3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:38.824661 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea2b83-7f2a-4544-aa1c-59895dc0b3f3 None admin] PATH_INFO: `/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a/groups/bded54c235c74f12a167629af5a6e94e/roles/57a070b43ad940f1886c69721dd304c2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:38.827057 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abea2b83-7f2a-4544-aa1c-59895dc0b3f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=a9d326f0c0fa4a38aa7ba8c8c6797c5a, group_id=bded54c235c74f12a167629af5a6e94e, role_id=57a070b43ad940f1886c69721dd304c2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:38.828511 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abea2b83-7f2a-4544-aa1c-59895dc0b3f3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:38.837378 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de50296c-32d7-444e-97b4-59c828414204 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:38.847296 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 197/390] 10.222.0.40 () {58 vars in 1442 bytes} [Fri Aug 5 10:13:38 2022] PUT /identity/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a/groups/bded54c235c74f12a167629af5a6e94e/roles/57a070b43ad940f1886c69721dd304c2 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:13:38.888953 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b76df6-dd02-45d6-b914-13f370e94911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:38.890124 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b76df6-dd02-45d6-b914-13f370e94911 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:38.890576 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b76df6-dd02-45d6-b914-13f370e94911 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:38.891116 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b76df6-dd02-45d6-b914-13f370e94911 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:38.892080 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b76df6-dd02-45d6-b914-13f370e94911 None admin] RBAC: Adding query filter params (name=member) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:38.892559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b76df6-dd02-45d6-b914-13f370e94911 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:38.894295 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b76df6-dd02-45d6-b914-13f370e94911 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:38.905892 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 194/391] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 10:13:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:38.912208 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea2b83-7f2a-4544-aa1c-59895dc0b3f3 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:38.916616 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:39.024346 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea2b83-7f2a-4544-aa1c-59895dc0b3f3 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:39.066782 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec441c5-653d-42e3-bd1e-503344c105e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:39.067774 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec441c5-653d-42e3-bd1e-503344c105e0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:39.068245 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec441c5-653d-42e3-bd1e-503344c105e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:39.068679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec441c5-653d-42e3-bd1e-503344c105e0 None admin] PATH_INFO: `/v3/groups/bded54c235c74f12a167629af5a6e94e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:39.070071 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec441c5-653d-42e3-bd1e-503344c105e0 None admin] RBAC: Authorizing `identity:get_group(group_id=bded54c235c74f12a167629af5a6e94e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:39.071741 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec441c5-653d-42e3-bd1e-503344c105e0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:39.073841 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 198/392] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 5 10:13:38 2022] GET /identity/v3/groups/bded54c235c74f12a167629af5a6e94e => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:39.079821 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b76df6-dd02-45d6-b914-13f370e94911 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:39.218224 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b76df6-dd02-45d6-b914-13f370e94911 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:39.223875 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c62a5c8-3a8d-4c6d-bf84-f09eb21bbcde None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:39.224292 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c62a5c8-3a8d-4c6d-bf84-f09eb21bbcde None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:39.224418 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c62a5c8-3a8d-4c6d-bf84-f09eb21bbcde None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:39.225848 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 199/393] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 10:13:39.229741 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:39.233708 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450f495-ce34-4201-becf-1e4ce54fc4fb None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:39.234006 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450f495-ce34-4201-becf-1e4ce54fc4fb None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:39.234274 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450f495-ce34-4201-becf-1e4ce54fc4fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:39.264078 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7702d6c-b893-412d-8bf3-7d3e0b3e0e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:39.265118 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7702d6c-b893-412d-8bf3-7d3e0b3e0e1f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:39.265588 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7702d6c-b893-412d-8bf3-7d3e0b3e0e1f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:39.266042 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7702d6c-b893-412d-8bf3-7d3e0b3e0e1f None admin] PATH_INFO: `/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:39.266383 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d450f495-ce34-4201-becf-1e4ce54fc4fb None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:39.267651 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7702d6c-b893-412d-8bf3-7d3e0b3e0e1f None admin] RBAC: Authorizing `identity:get_project(project_id=a9d326f0c0fa4a38aa7ba8c8c6797c5a)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:39.267754 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d450f495-ce34-4201-becf-1e4ce54fc4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:39.269369 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7702d6c-b893-412d-8bf3-7d3e0b3e0e1f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:39.271831 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 195/394] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:13:39 2022] GET /identity/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:39.277599 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7702d6c-b893-412d-8bf3-7d3e0b3e0e1f None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:39.382247 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 200/395] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:39.407148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450f495-ce34-4201-becf-1e4ce54fc4fb None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:39.409461 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:39.430146 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7702d6c-b893-412d-8bf3-7d3e0b3e0e1f None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:39.470337 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb4dbe6-0b96-47ec-b210-40938961b0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:39.471102 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4dbe6-0b96-47ec-b210-40938961b0c4 None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:39.471394 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4dbe6-0b96-47ec-b210-40938961b0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:39.471670 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4dbe6-0b96-47ec-b210-40938961b0c4 None admin] PATH_INFO: `/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a/groups/bded54c235c74f12a167629af5a6e94e/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:39.473911 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb4dbe6-0b96-47ec-b210-40938961b0c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=a9d326f0c0fa4a38aa7ba8c8c6797c5a, group_id=bded54c235c74f12a167629af5a6e94e, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:39.477692 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb4dbe6-0b96-47ec-b210-40938961b0c4 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:39.501444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 196/396] 10.222.0.40 () {58 vars in 1442 bytes} [Fri Aug 5 10:13:39 2022] PUT /identity/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a/groups/bded54c235c74f12a167629af5a6e94e/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:13:39.517266 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450f495-ce34-4201-becf-1e4ce54fc4fb None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:39.556676 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c432d5-b9da-4d2d-b683-5fb6d14007d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:39.557358 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c432d5-b9da-4d2d-b683-5fb6d14007d8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:39.557633 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c432d5-b9da-4d2d-b683-5fb6d14007d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:39.557900 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c432d5-b9da-4d2d-b683-5fb6d14007d8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:39.571451 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c432d5-b9da-4d2d-b683-5fb6d14007d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:39.576692 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-f9c432d5-b9da-4d2d-b683-5fb6d14007d8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 10:13:39.576692 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:39.576692 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:39.576692 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:39.576692 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:39.576692 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:39.576692 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:39.576692 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:39.576692 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:39.576692 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:39.576692 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:39.576692 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:39.576692 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:39.576692 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:39.576692 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:39.576692 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:39.577680 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:39.577680 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:39.577680 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:39.577680 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:39.577680 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:39.577680 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:39.577680 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:39.577680 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:39.577680 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:39.577680 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:39.577680 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:39.577680 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:39.577680 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:39.577680 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:39.577680 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:39.577680 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:39.578466 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:39.578466 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:39.578466 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:39.578466 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:39.578466 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:39.578466 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:39.578466 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:39.578466 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:39.578466 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:39.578466 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:39.578466 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 10:13:39.578466 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:39.579128 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 201/397] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:13:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:13:39.588110 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb4dbe6-0b96-47ec-b210-40938961b0c4 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:39.643235 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:39.728641 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb4dbe6-0b96-47ec-b210-40938961b0c4 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:39.766782 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e348bcf9-4617-493f-987a-082cb796f2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:39.767463 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e348bcf9-4617-493f-987a-082cb796f2d9 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:39.767746 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e348bcf9-4617-493f-987a-082cb796f2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:39.768020 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e348bcf9-4617-493f-987a-082cb796f2d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:39.768754 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e348bcf9-4617-493f-987a-082cb796f2d9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:39.769060 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e348bcf9-4617-493f-987a-082cb796f2d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:39.770487 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e348bcf9-4617-493f-987a-082cb796f2d9 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:39.782039 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 197/398] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 10:13:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:39.787298 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c432d5-b9da-4d2d-b683-5fb6d14007d8 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:39.872961 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:39.916722 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c432d5-b9da-4d2d-b683-5fb6d14007d8 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:39.953540 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44461ea1-90b4-48e6-992a-f1988b59201c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:39.954207 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44461ea1-90b4-48e6-992a-f1988b59201c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:39.954504 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44461ea1-90b4-48e6-992a-f1988b59201c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:39.954762 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44461ea1-90b4-48e6-992a-f1988b59201c None admin] PATH_INFO: `/v3/groups/bded54c235c74f12a167629af5a6e94e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:39.955962 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44461ea1-90b4-48e6-992a-f1988b59201c None admin] RBAC: Authorizing `identity:get_group(group_id=bded54c235c74f12a167629af5a6e94e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:39.957430 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44461ea1-90b4-48e6-992a-f1988b59201c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:39.959429 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 202/399] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 5 10:13:39 2022] GET /identity/v3/groups/bded54c235c74f12a167629af5a6e94e => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:39.964898 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e348bcf9-4617-493f-987a-082cb796f2d9 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:40.074285 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e348bcf9-4617-493f-987a-082cb796f2d9 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:40.077580 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:40.126628 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0bbb6d-81ac-43ea-b09b-61784d807b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:40.127312 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0bbb6d-81ac-43ea-b09b-61784d807b48 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:40.127589 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0bbb6d-81ac-43ea-b09b-61784d807b48 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:40.127866 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0bbb6d-81ac-43ea-b09b-61784d807b48 None admin] PATH_INFO: `/v3/projects/400a983b4cd947d487b7fa86a5f5c339` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:40.129067 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0bbb6d-81ac-43ea-b09b-61784d807b48 None admin] RBAC: Authorizing `identity:get_project(project_id=400a983b4cd947d487b7fa86a5f5c339)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:40.130521 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0bbb6d-81ac-43ea-b09b-61784d807b48 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:40.132881 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 198/400] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:13:39 2022] GET /identity/v3/projects/400a983b4cd947d487b7fa86a5f5c339 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:40.139010 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44461ea1-90b4-48e6-992a-f1988b59201c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:40.254842 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207dfdb0-b5f9-4e23-979a-60bdc61f8507 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:40.255394 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207dfdb0-b5f9-4e23-979a-60bdc61f8507 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:40.255394 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207dfdb0-b5f9-4e23-979a-60bdc61f8507 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:40.257058 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 199/401] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:40.263147 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715448b-2acf-481c-a9c2-56d763fa38de None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:40.263433 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715448b-2acf-481c-a9c2-56d763fa38de None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:40.263706 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715448b-2acf-481c-a9c2-56d763fa38de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:40.286147 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-8715448b-2acf-481c-a9c2-56d763fa38de None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:40.287386 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8715448b-2acf-481c-a9c2-56d763fa38de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:40.314564 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44461ea1-90b4-48e6-992a-f1988b59201c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:40.344592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:40.354965 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d676d2f9-f0d8-4588-aaa1-e20ce686ccb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:40.355735 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d676d2f9-f0d8-4588-aaa1-e20ce686ccb8 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:40.358922 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d676d2f9-f0d8-4588-aaa1-e20ce686ccb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:40.358922 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d676d2f9-f0d8-4588-aaa1-e20ce686ccb8 None admin] PATH_INFO: `/v3/projects/400a983b4cd947d487b7fa86a5f5c339/groups/bded54c235c74f12a167629af5a6e94e/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:40.358922 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d676d2f9-f0d8-4588-aaa1-e20ce686ccb8 None admin] RBAC: Authorizing `identity:create_grant(project_id=400a983b4cd947d487b7fa86a5f5c339, group_id=bded54c235c74f12a167629af5a6e94e, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:40.361023 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d676d2f9-f0d8-4588-aaa1-e20ce686ccb8 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:40.390561 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 203/402] 10.222.0.40 () {58 vars in 1442 bytes} [Fri Aug 5 10:13:40 2022] PUT /identity/v3/projects/400a983b4cd947d487b7fa86a5f5c339/groups/bded54c235c74f12a167629af5a6e94e/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:13:40.402875 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 200/403] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:40.433036 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d676d2f9-f0d8-4588-aaa1-e20ce686ccb8 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:40.543571 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbdd8a5-a63b-4921-95da-a6f162ceb91b None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:40.543877 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbdd8a5-a63b-4921-95da-a6f162ceb91b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:40.544110 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbdd8a5-a63b-4921-95da-a6f162ceb91b None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:40.546189 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 201/404] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:40.551784 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3353bb1-9d92-4b05-a7c3-3d019b237cce None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:40.552056 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3353bb1-9d92-4b05-a7c3-3d019b237cce None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:40.552331 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3353bb1-9d92-4b05-a7c3-3d019b237cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:40.573082 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-e3353bb1-9d92-4b05-a7c3-3d019b237cce None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:40.574325 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e3353bb1-9d92-4b05-a7c3-3d019b237cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:40.576736 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d676d2f9-f0d8-4588-aaa1-e20ce686ccb8 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:40.630576 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304efe6e-0de6-4832-aab8-43039ea9697a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:40.631229 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304efe6e-0de6-4832-aab8-43039ea9697a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:40.631511 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304efe6e-0de6-4832-aab8-43039ea9697a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:40.631789 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304efe6e-0de6-4832-aab8-43039ea9697a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:40.643443 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304efe6e-0de6-4832-aab8-43039ea9697a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:40.648449 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-304efe6e-0de6-4832-aab8-43039ea9697a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 10:13:40.648449 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:40.648449 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:40.648449 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:40.648449 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:40.648449 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:40.648449 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:40.648449 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:40.648449 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:40.648449 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:40.648449 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:40.648449 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:40.648449 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:40.648449 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:40.648449 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:40.648449 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:40.649384 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:40.649384 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:40.649384 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:40.649384 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:40.649384 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:40.649384 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:40.649384 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:40.649384 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:40.649384 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:40.649384 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:40.649384 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:40.649384 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:40.649384 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:40.649384 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:40.649384 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:40.649384 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:40.650290 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:40.650290 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:40.650290 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:40.650290 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:40.650290 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:40.650290 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:40.650290 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:40.650290 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:40.650290 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:40.650290 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:40.650290 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 10:13:40.650290 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:40.650290 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 204/405] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 5 10:13:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:40.655780 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304efe6e-0de6-4832-aab8-43039ea9697a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:40.660311 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 202/406] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 10:13:40.683455 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3353bb1-9d92-4b05-a7c3-3d019b237cce None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:40.708857 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:40.729145 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:40.771793 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304efe6e-0de6-4832-aab8-43039ea9697a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:40.822150 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f49c6-625e-485f-a192-773c17d7fa0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:40.822715 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f49c6-625e-485f-a192-773c17d7fa0c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:40.823034 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f49c6-625e-485f-a192-773c17d7fa0c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:40.823308 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f49c6-625e-485f-a192-773c17d7fa0c None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:40.824055 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f49c6-625e-485f-a192-773c17d7fa0c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:40.824357 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f49c6-625e-485f-a192-773c17d7fa0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:40.825790 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f49c6-625e-485f-a192-773c17d7fa0c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:40.836166 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 205/407] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:13:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:13:40.841767 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f49c6-625e-485f-a192-773c17d7fa0c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:40.892322 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3353bb1-9d92-4b05-a7c3-3d019b237cce None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:40.968809 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f49c6-625e-485f-a192-773c17d7fa0c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:40.970758 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b21134-81cc-49ba-9f41-f6ead2c14f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:40.971663 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b21134-81cc-49ba-9f41-f6ead2c14f41 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:40.972164 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b21134-81cc-49ba-9f41-f6ead2c14f41 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:40.972627 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b21134-81cc-49ba-9f41-f6ead2c14f41 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:40.982566 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:40.993992 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b21134-81cc-49ba-9f41-f6ead2c14f41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:40.999342 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-f5b21134-81cc-49ba-9f41-f6ead2c14f41 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:13:40.999342 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:40.999342 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:40.999342 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:40.999342 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:40.999342 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:40.999342 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:40.999342 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:40.999342 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:40.999342 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:40.999342 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:40.999342 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:40.999342 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:40.999342 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:40.999342 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:40.999342 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:41.000326 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:41.000326 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:41.000326 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:41.000326 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:41.000326 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:41.000326 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:41.000326 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:41.000326 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:41.000326 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:41.000326 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:41.000326 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:41.000326 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:41.000326 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:41.000326 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:41.000326 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:41.000326 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:41.001128 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:41.001128 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:41.001128 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:41.001128 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:41.001128 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:41.001128 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:41.001128 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:41.001128 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:41.001128 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:41.001128 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:41.001128 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:13:41.001128 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:13:41.001128 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 203/408] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 10:13:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:13:41.007962 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b21134-81cc-49ba-9f41-f6ead2c14f41 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:41.017870 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdbbdad-9086-4e62-a8b9-6de03271a23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:41.019160 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdbbdad-9086-4e62-a8b9-6de03271a23d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:41.019614 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdbbdad-9086-4e62-a8b9-6de03271a23d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:41.022569 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdbbdad-9086-4e62-a8b9-6de03271a23d None admin] PATH_INFO: `/v3/groups/bded54c235c74f12a167629af5a6e94e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:41.024049 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdbbdad-9086-4e62-a8b9-6de03271a23d None admin] RBAC: Authorizing `identity:get_group(group_id=bded54c235c74f12a167629af5a6e94e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:41.025648 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdbbdad-9086-4e62-a8b9-6de03271a23d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:41.029133 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 206/409] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 5 10:13:40 2022] GET /identity/v3/groups/bded54c235c74f12a167629af5a6e94e => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:13:41.044048 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdbbdad-9086-4e62-a8b9-6de03271a23d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:41.178352 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:41.194026 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdbbdad-9086-4e62-a8b9-6de03271a23d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:41.214502 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b21134-81cc-49ba-9f41-f6ead2c14f41 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:41.231253 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8d4d1c-fa46-4a98-a65f-1259f3f5d4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:41.232323 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8d4d1c-fa46-4a98-a65f-1259f3f5d4d0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:41.233526 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8d4d1c-fa46-4a98-a65f-1259f3f5d4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:41.234015 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8d4d1c-fa46-4a98-a65f-1259f3f5d4d0 None admin] PATH_INFO: `/v3/projects/400a983b4cd947d487b7fa86a5f5c339` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:41.235514 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8d4d1c-fa46-4a98-a65f-1259f3f5d4d0 None admin] RBAC: Authorizing `identity:get_project(project_id=400a983b4cd947d487b7fa86a5f5c339)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:41.237193 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8d4d1c-fa46-4a98-a65f-1259f3f5d4d0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:41.239442 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 207/410] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:13:41 2022] GET /identity/v3/projects/400a983b4cd947d487b7fa86a5f5c339 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:13:41.251962 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8d4d1c-fa46-4a98-a65f-1259f3f5d4d0 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:41.271240 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa1ab36-758e-4154-af28-35e3fe10e4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:41.271791 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1ab36-758e-4154-af28-35e3fe10e4b9 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:41.272186 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1ab36-758e-4154-af28-35e3fe10e4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:41.272594 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1ab36-758e-4154-af28-35e3fe10e4b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:41.273408 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa1ab36-758e-4154-af28-35e3fe10e4b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:41.273806 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa1ab36-758e-4154-af28-35e3fe10e4b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:41.279525 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa1ab36-758e-4154-af28-35e3fe10e4b9 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:41.298809 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 204/411] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:13:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:41.304086 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa1ab36-758e-4154-af28-35e3fe10e4b9 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:41.313940 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:41.368973 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8d4d1c-fa46-4a98-a65f-1259f3f5d4d0 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:41.406495 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a99d9d7-4d81-49ac-95e9-bd857972869d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:41.407582 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a99d9d7-4d81-49ac-95e9-bd857972869d None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:41.408082 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a99d9d7-4d81-49ac-95e9-bd857972869d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:41.408558 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a99d9d7-4d81-49ac-95e9-bd857972869d None admin] PATH_INFO: `/v3/projects/400a983b4cd947d487b7fa86a5f5c339/groups/bded54c235c74f12a167629af5a6e94e/roles/57a070b43ad940f1886c69721dd304c2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:41.411054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a99d9d7-4d81-49ac-95e9-bd857972869d None admin] RBAC: Authorizing `identity:create_grant(project_id=400a983b4cd947d487b7fa86a5f5c339, group_id=bded54c235c74f12a167629af5a6e94e, role_id=57a070b43ad940f1886c69721dd304c2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:41.412945 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a99d9d7-4d81-49ac-95e9-bd857972869d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:41.431153 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 208/412] 10.222.0.40 () {58 vars in 1442 bytes} [Fri Aug 5 10:13:41 2022] PUT /identity/v3/projects/400a983b4cd947d487b7fa86a5f5c339/groups/bded54c235c74f12a167629af5a6e94e/roles/57a070b43ad940f1886c69721dd304c2 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:13:41.508658 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa1ab36-758e-4154-af28-35e3fe10e4b9 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:41.554680 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2a932c-d226-47c2-9b30-abdedd643b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:41.555408 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a932c-d226-47c2-9b30-abdedd643b18 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:41.555693 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a932c-d226-47c2-9b30-abdedd643b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:41.555971 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a932c-d226-47c2-9b30-abdedd643b18 None admin] PATH_INFO: `/v3/groups/f8ba520a72c14d3f9de2539d0c047a2c` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:41.557224 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2a932c-d226-47c2-9b30-abdedd643b18 None admin] RBAC: Authorizing `identity:get_group(group_id=f8ba520a72c14d3f9de2539d0c047a2c)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:41.558743 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2a932c-d226-47c2-9b30-abdedd643b18 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:41.561743 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 205/413] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 5 10:13:41 2022] GET /identity/v3/groups/f8ba520a72c14d3f9de2539d0c047a2c => generated 229 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:41.566525 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a99d9d7-4d81-49ac-95e9-bd857972869d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:41.728043 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a99d9d7-4d81-49ac-95e9-bd857972869d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:41.740003 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:41.767020 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7b4c0d-e5eb-414d-b769-758cbc0800e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:41.768024 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b4c0d-e5eb-414d-b769-758cbc0800e7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:41.768491 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b4c0d-e5eb-414d-b769-758cbc0800e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:41.768963 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b4c0d-e5eb-414d-b769-758cbc0800e7 None admin] PATH_INFO: `/v3/projects/be89c43472bc49a2bd0e46cddf175958` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:41.770860 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7b4c0d-e5eb-414d-b769-758cbc0800e7 None admin] RBAC: Authorizing `identity:get_project(project_id=be89c43472bc49a2bd0e46cddf175958)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:41.772543 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7b4c0d-e5eb-414d-b769-758cbc0800e7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:41.776208 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 209/414] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:13:41 2022] GET /identity/v3/projects/be89c43472bc49a2bd0e46cddf175958 => generated 344 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:41.781742 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2a932c-d226-47c2-9b30-abdedd643b18 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:41.820047 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:41.945432 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2a932c-d226-47c2-9b30-abdedd643b18 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:41.984908 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b4c47-add5-4b80-b0b0-7f24dcef9f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:41.985503 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b4c47-add5-4b80-b0b0-7f24dcef9f59 None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:41.985782 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b4c47-add5-4b80-b0b0-7f24dcef9f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:41.986051 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b4c47-add5-4b80-b0b0-7f24dcef9f59 None admin] PATH_INFO: `/v3/projects/be89c43472bc49a2bd0e46cddf175958/groups/f8ba520a72c14d3f9de2539d0c047a2c/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:41.992236 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5b4c47-add5-4b80-b0b0-7f24dcef9f59 None admin] RBAC: Authorizing `identity:create_grant(project_id=be89c43472bc49a2bd0e46cddf175958, group_id=f8ba520a72c14d3f9de2539d0c047a2c, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:41.993697 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5b4c47-add5-4b80-b0b0-7f24dcef9f59 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:42.007771 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60464a-7017-405e-b9c2-2c53b46cd477 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:42.008073 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60464a-7017-405e-b9c2-2c53b46cd477 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:42.008512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60464a-7017-405e-b9c2-2c53b46cd477 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:42.010078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 210/415] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 10:13:42.016663 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca22468-00f7-4586-98fe-d6d3c1f9fc1f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:42.017266 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca22468-00f7-4586-98fe-d6d3c1f9fc1f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:42.017557 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca22468-00f7-4586-98fe-d6d3c1f9fc1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:42.022449 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 206/416] 10.222.0.40 () {58 vars in 1442 bytes} [Fri Aug 5 10:13:41 2022] PUT /identity/v3/projects/be89c43472bc49a2bd0e46cddf175958/groups/f8ba520a72c14d3f9de2539d0c047a2c/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:13:42.040183 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1ca22468-00f7-4586-98fe-d6d3c1f9fc1f None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:42.041462 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1ca22468-00f7-4586-98fe-d6d3c1f9fc1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:42.120721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 211/417] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 10:13:42.145437 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b4c47-add5-4b80-b0b0-7f24dcef9f59 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:42.240395 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:42.260065 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b4c47-add5-4b80-b0b0-7f24dcef9f59 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:42.299311 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ea685e-2ef6-42b8-a6dc-37f93e5893c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:42.299972 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea685e-2ef6-42b8-a6dc-37f93e5893c0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:42.300254 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea685e-2ef6-42b8-a6dc-37f93e5893c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:42.300534 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea685e-2ef6-42b8-a6dc-37f93e5893c0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:42.313006 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ea685e-2ef6-42b8-a6dc-37f93e5893c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:42.318755 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-11ea685e-2ef6-42b8-a6dc-37f93e5893c0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 10:13:42.318755 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:42.318755 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:42.318755 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:42.318755 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:42.318755 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:42.318755 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:42.318755 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:42.318755 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:42.318755 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:42.318755 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:42.318755 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:42.318755 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:42.318755 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:42.318755 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:42.318755 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:42.319660 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:42.319660 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:42.319660 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:42.319660 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:42.319660 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:42.319660 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:42.319660 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:42.319660 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:42.319660 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:42.319660 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:42.319660 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:42.319660 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:42.319660 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:42.319660 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:42.319660 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:42.319660 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:42.320458 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:42.320458 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:42.320458 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:42.320458 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:42.320458 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:42.320458 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:42.320458 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:42.320458 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:42.320458 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:42.320458 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:42.320458 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 10:13:42.320458 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:13:42.320458 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 207/418] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 5 10:13:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:42.326174 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca22468-00f7-4586-98fe-d6d3c1f9fc1f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:42.435810 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca22468-00f7-4586-98fe-d6d3c1f9fc1f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:42.445131 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:42.474004 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e70f39-80e7-4d97-9ebc-1fcc390ec748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:42.474647 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e70f39-80e7-4d97-9ebc-1fcc390ec748 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:42.474972 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e70f39-80e7-4d97-9ebc-1fcc390ec748 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:42.475256 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e70f39-80e7-4d97-9ebc-1fcc390ec748 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:42.475978 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e70f39-80e7-4d97-9ebc-1fcc390ec748 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:42.476281 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e70f39-80e7-4d97-9ebc-1fcc390ec748 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:42.477730 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e70f39-80e7-4d97-9ebc-1fcc390ec748 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:42.487749 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 212/419] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:13:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:42.493504 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ea685e-2ef6-42b8-a6dc-37f93e5893c0 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:42.546105 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3964ee-9e36-4afe-93f4-fffd4c336819 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:42.546418 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3964ee-9e36-4afe-93f4-fffd4c336819 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:42.546655 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3964ee-9e36-4afe-93f4-fffd4c336819 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:42.548109 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 213/420] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:42.553596 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a3d2c-3506-4dc3-b820-a8697c2c10c7 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:42.553875 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a3d2c-3506-4dc3-b820-a8697c2c10c7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:42.554146 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a3d2c-3506-4dc3-b820-a8697c2c10c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:42.596266 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-737a3d2c-3506-4dc3-b820-a8697c2c10c7 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:42.597878 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-737a3d2c-3506-4dc3-b820-a8697c2c10c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:42.610749 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ea685e-2ef6-42b8-a6dc-37f93e5893c0 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:42.650438 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66211cd7-5fac-4f13-bf83-0f32e968cfea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:42.651114 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66211cd7-5fac-4f13-bf83-0f32e968cfea None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:42.651398 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66211cd7-5fac-4f13-bf83-0f32e968cfea None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:42.651670 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66211cd7-5fac-4f13-bf83-0f32e968cfea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:42.654725 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:42.710132 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 214/421] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 10:13:42.712068 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82378646-d4c9-4d53-b1cb-2ec0ad166595 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:42.712559 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82378646-d4c9-4d53-b1cb-2ec0ad166595 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:42.713013 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82378646-d4c9-4d53-b1cb-2ec0ad166595 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:42.718927 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 215/422] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:42.720356 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe86330-afee-4f35-81ff-c4e329fe2dea None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:42.720822 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe86330-afee-4f35-81ff-c4e329fe2dea None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:42.721373 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe86330-afee-4f35-81ff-c4e329fe2dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:42.736477 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66211cd7-5fac-4f13-bf83-0f32e968cfea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:42.738207 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66211cd7-5fac-4f13-bf83-0f32e968cfea None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:42.741557 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-4fe86330-afee-4f35-81ff-c4e329fe2dea None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:42.742942 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4fe86330-afee-4f35-81ff-c4e329fe2dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:42.815743 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 216/423] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:42.821883 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe86330-afee-4f35-81ff-c4e329fe2dea None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:42.857390 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:42.869450 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 208/424] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:13:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 10:13:42.875488 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66211cd7-5fac-4f13-bf83-0f32e968cfea None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:42.909593 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:42.966148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe86330-afee-4f35-81ff-c4e329fe2dea None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:42.995279 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66211cd7-5fac-4f13-bf83-0f32e968cfea None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:43.013455 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a1185-a84b-4705-befd-e690af401a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:43.014615 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a1185-a84b-4705-befd-e690af401a31 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:43.014895 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a1185-a84b-4705-befd-e690af401a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:43.015452 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a1185-a84b-4705-befd-e690af401a31 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:43.029629 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6a1185-a84b-4705-befd-e690af401a31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:43.035757 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-8e6a1185-a84b-4705-befd-e690af401a31 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 10:13:43.035757 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:43.035757 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:43.035757 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:43.035757 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:43.035757 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:43.035757 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:43.035757 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:43.035757 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:43.035757 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:43.035757 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:43.035757 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:43.035757 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:43.035757 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:43.035757 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:43.035757 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:43.036842 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:43.036842 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:43.036842 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:43.036842 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:43.036842 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:43.036842 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:43.036842 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:43.036842 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:43.036842 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:43.036842 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:43.036842 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:43.036842 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:43.036842 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:43.036842 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:43.036842 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:43.036842 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:43.037826 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:43.037826 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:43.037826 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:43.037826 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:43.037826 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:43.037826 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:43.037826 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:43.037826 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:43.037826 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:43.037826 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:43.037826 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 10:13:43.037826 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:43.038564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a86ba7-c35f-4104-a412-ac80ad04ce29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:43.038564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a86ba7-c35f-4104-a412-ac80ad04ce29 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:43.038564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a86ba7-c35f-4104-a412-ac80ad04ce29 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:43.039001 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a86ba7-c35f-4104-a412-ac80ad04ce29 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:43.042253 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 217/425] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:13:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:43.043547 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a1185-a84b-4705-befd-e690af401a31 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:43.053073 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a86ba7-c35f-4104-a412-ac80ad04ce29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:43.059688 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-75a86ba7-c35f-4104-a412-ac80ad04ce29 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 10:13:43.059688 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:43.059688 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:43.059688 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:43.059688 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:43.059688 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:43.059688 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:43.059688 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:43.059688 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:43.059688 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:43.059688 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:43.059688 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:43.059688 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:43.059688 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:43.059688 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:43.059688 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:43.062149 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:43.062149 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:43.062149 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:43.062149 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:43.062149 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:43.062149 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:43.062149 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:43.062149 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:43.062149 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:43.062149 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:43.062149 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:43.062149 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:43.062149 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:43.062149 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:43.062149 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:43.062149 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:43.063368 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:43.063368 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:43.063368 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:43.063368 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:43.063368 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:43.063368 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:43.063368 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:43.063368 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:43.063368 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:43.063368 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:43.063368 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 10:13:43.063368 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:13:43.063368 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 209/426] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:13:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:43.064730 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a86ba7-c35f-4104-a412-ac80ad04ce29 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:43.147972 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:43.165855 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:43.173073 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a86ba7-c35f-4104-a412-ac80ad04ce29 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:43.174239 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a1185-a84b-4705-befd-e690af401a31 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:43.207532 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e0edf-63d9-4624-87f2-854dbb88aa6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:43.208283 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e0edf-63d9-4624-87f2-854dbb88aa6c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:43.208656 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e0edf-63d9-4624-87f2-854dbb88aa6c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:43.209019 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e0edf-63d9-4624-87f2-854dbb88aa6c None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:43.209832 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835e0edf-63d9-4624-87f2-854dbb88aa6c None admin] RBAC: Adding query filter params (name=member) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:43.210219 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835e0edf-63d9-4624-87f2-854dbb88aa6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:43.211732 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835e0edf-63d9-4624-87f2-854dbb88aa6c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:43.212326 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edca7bb-1426-4192-90ba-c49f3b1be257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:43.213242 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edca7bb-1426-4192-90ba-c49f3b1be257 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:43.213673 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edca7bb-1426-4192-90ba-c49f3b1be257 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:43.214239 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edca7bb-1426-4192-90ba-c49f3b1be257 None admin] PATH_INFO: `/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:43.215745 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edca7bb-1426-4192-90ba-c49f3b1be257 None admin] RBAC: Authorizing `identity:get_project(project_id=a9d326f0c0fa4a38aa7ba8c8c6797c5a)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:43.217546 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edca7bb-1426-4192-90ba-c49f3b1be257 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:43.219779 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 218/427] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:13:43 2022] GET /identity/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:43.220722 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 210/428] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 10:13:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:43.221106 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edca7bb-1426-4192-90ba-c49f3b1be257 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:43.226069 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e0edf-63d9-4624-87f2-854dbb88aa6c None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:43.324097 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e0edf-63d9-4624-87f2-854dbb88aa6c None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:43.328174 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edca7bb-1426-4192-90ba-c49f3b1be257 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:43.358530 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cfdd2d-3091-463e-a7af-0ad516115773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:43.359410 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cfdd2d-3091-463e-a7af-0ad516115773 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:43.359760 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cfdd2d-3091-463e-a7af-0ad516115773 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:43.360100 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cfdd2d-3091-463e-a7af-0ad516115773 None admin] PATH_INFO: `/v3/groups/bded54c235c74f12a167629af5a6e94e` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:43.361438 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cfdd2d-3091-463e-a7af-0ad516115773 None admin] RBAC: Authorizing `identity:get_group(group_id=bded54c235c74f12a167629af5a6e94e)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:43.363014 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cfdd2d-3091-463e-a7af-0ad516115773 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:43.369374 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 211/429] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 5 10:13:43 2022] GET /identity/v3/groups/bded54c235c74f12a167629af5a6e94e => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:43.369374 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cfdd2d-3091-463e-a7af-0ad516115773 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:43.371628 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e681d7-9c9f-4477-86ea-1f53a57e6bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:43.372622 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e681d7-9c9f-4477-86ea-1f53a57e6bba None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:43.373270 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e681d7-9c9f-4477-86ea-1f53a57e6bba None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:43.373750 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e681d7-9c9f-4477-86ea-1f53a57e6bba None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:43.374679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e681d7-9c9f-4477-86ea-1f53a57e6bba None admin] RBAC: Adding query filter params (name=member) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:43.375231 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e681d7-9c9f-4477-86ea-1f53a57e6bba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:43.376863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e681d7-9c9f-4477-86ea-1f53a57e6bba None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:43.386933 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 219/430] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 10:13:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:43.388096 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e681d7-9c9f-4477-86ea-1f53a57e6bba None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:43.396644 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:43.412091 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:43.479011 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cfdd2d-3091-463e-a7af-0ad516115773 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:43.500388 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e681d7-9c9f-4477-86ea-1f53a57e6bba None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:43.516870 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46093c3f-6cef-44cc-bd59-9701097a8f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:43.517517 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46093c3f-6cef-44cc-bd59-9701097a8f20 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:43.517807 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46093c3f-6cef-44cc-bd59-9701097a8f20 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:43.518084 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46093c3f-6cef-44cc-bd59-9701097a8f20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:43.541007 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4f7618-8b1f-4369-87e1-1a5d171ae07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:43.542052 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f7618-8b1f-4369-87e1-1a5d171ae07c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:43.542586 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f7618-8b1f-4369-87e1-1a5d171ae07c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:43.543101 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f7618-8b1f-4369-87e1-1a5d171ae07c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:43.544136 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4f7618-8b1f-4369-87e1-1a5d171ae07c None admin] RBAC: Adding query filter params (group.id=bded54c235c74f12a167629af5a6e94e, role.id=57a070b43ad940f1886c69721dd304c2, scope.project.id=a9d326f0c0fa4a38aa7ba8c8c6797c5a) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:43.545116 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4f7618-8b1f-4369-87e1-1a5d171ae07c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:43.547033 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4f7618-8b1f-4369-87e1-1a5d171ae07c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:43.559050 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 220/431] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Aug 5 10:13:43 2022] GET /identity/v3/role_assignments?group.id=bded54c235c74f12a167629af5a6e94e&scope.project.id=a9d326f0c0fa4a38aa7ba8c8c6797c5a&role.id=57a070b43ad940f1886c69721dd304c2 => generated 611 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:43.560383 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4f7618-8b1f-4369-87e1-1a5d171ae07c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:43.593689 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46093c3f-6cef-44cc-bd59-9701097a8f20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:43.595212 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46093c3f-6cef-44cc-bd59-9701097a8f20 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:43.622121 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:43.707225 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 212/432] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:13:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 10:13:43.712743 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46093c3f-6cef-44cc-bd59-9701097a8f20 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:43.717307 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:43.754981 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4f7618-8b1f-4369-87e1-1a5d171ae07c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:43.800615 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e2813b-fae4-4e4e-9a40-ddcead1e6533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:43.801261 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2813b-fae4-4e4e-9a40-ddcead1e6533 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:43.801547 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2813b-fae4-4e4e-9a40-ddcead1e6533 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:43.801827 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2813b-fae4-4e4e-9a40-ddcead1e6533 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:43.860055 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46093c3f-6cef-44cc-bd59-9701097a8f20 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:43.878090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e2813b-fae4-4e4e-9a40-ddcead1e6533 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:43.879630 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e2813b-fae4-4e4e-9a40-ddcead1e6533 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:43.887325 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:43.902419 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33614326-9685-4133-98c2-3131adee96ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:43.903218 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33614326-9685-4133-98c2-3131adee96ec None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:43.903565 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33614326-9685-4133-98c2-3131adee96ec None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:43.903908 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33614326-9685-4133-98c2-3131adee96ec None admin] PATH_INFO: `/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:43.906478 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33614326-9685-4133-98c2-3131adee96ec None admin] RBAC: Authorizing `identity:get_project(project_id=a9d326f0c0fa4a38aa7ba8c8c6797c5a)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:43.908088 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33614326-9685-4133-98c2-3131adee96ec None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:43.910999 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 213/433] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:13:43 2022] GET /identity/v3/projects/a9d326f0c0fa4a38aa7ba8c8c6797c5a => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:13:43.916374 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33614326-9685-4133-98c2-3131adee96ec None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:43.972848 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:43.992578 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 221/434] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:13:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 10:13:43.998963 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39571110-9918-4c3c-84ba-a71be6c11105 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:43.998963 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39571110-9918-4c3c-84ba-a71be6c11105 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:43.998963 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39571110-9918-4c3c-84ba-a71be6c11105 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:44.001574 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 222/435] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 10:13:44.002559 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39571110-9918-4c3c-84ba-a71be6c11105 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:44.043682 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33614326-9685-4133-98c2-3131adee96ec None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:44.078338 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d205ea72-e940-4292-a7e2-e9f9a745ba7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:44.079094 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d205ea72-e940-4292-a7e2-e9f9a745ba7e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:44.079291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d205ea72-e940-4292-a7e2-e9f9a745ba7e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:44.079558 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d205ea72-e940-4292-a7e2-e9f9a745ba7e None admin] PATH_INFO: `/v3/groups/bded54c235c74f12a167629af5a6e94e` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:44.080720 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d205ea72-e940-4292-a7e2-e9f9a745ba7e None admin] RBAC: Authorizing `identity:get_group(group_id=bded54c235c74f12a167629af5a6e94e)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:44.082169 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d205ea72-e940-4292-a7e2-e9f9a745ba7e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:44.083815 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 214/436] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 5 10:13:43 2022] GET /identity/v3/groups/bded54c235c74f12a167629af5a6e94e => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:13:44.085049 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d82ca-96ed-4421-9b15-1de62723c5cb None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:44.085336 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d82ca-96ed-4421-9b15-1de62723c5cb None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:44.085614 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d82ca-96ed-4421-9b15-1de62723c5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:44.104047 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-0d2d82ca-96ed-4421-9b15-1de62723c5cb None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:44.105305 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0d2d82ca-96ed-4421-9b15-1de62723c5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:44.119549 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39571110-9918-4c3c-84ba-a71be6c11105 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:44.163838 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7e60b6-4008-477c-9008-6ee388cf2430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:44.164502 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e60b6-4008-477c-9008-6ee388cf2430 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:44.164782 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e60b6-4008-477c-9008-6ee388cf2430 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:44.165058 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e60b6-4008-477c-9008-6ee388cf2430 None admin] PATH_INFO: `/v3/projects/400a983b4cd947d487b7fa86a5f5c339` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:44.166687 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7e60b6-4008-477c-9008-6ee388cf2430 None admin] RBAC: Authorizing `identity:get_project(project_id=400a983b4cd947d487b7fa86a5f5c339)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:44.168186 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7e60b6-4008-477c-9008-6ee388cf2430 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:44.169134 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:44.175379 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 223/437] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:13:44 2022] GET /identity/v3/projects/400a983b4cd947d487b7fa86a5f5c339 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:44.175379 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7e60b6-4008-477c-9008-6ee388cf2430 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:44.186337 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 215/438] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:44.187088 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2d82ca-96ed-4421-9b15-1de62723c5cb None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:44.244757 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:44.293923 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2d82ca-96ed-4421-9b15-1de62723c5cb None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:44.300214 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7e60b6-4008-477c-9008-6ee388cf2430 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:44.329300 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f12ca6e-62d6-40a2-bc9c-87da3f7af1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:44.329965 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12ca6e-62d6-40a2-bc9c-87da3f7af1d8 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:44.330239 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12ca6e-62d6-40a2-bc9c-87da3f7af1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:44.330513 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12ca6e-62d6-40a2-bc9c-87da3f7af1d8 None admin] PATH_INFO: `/v3/groups/bded54c235c74f12a167629af5a6e94e` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:44.331697 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f12ca6e-62d6-40a2-bc9c-87da3f7af1d8 None admin] RBAC: Authorizing `identity:get_group(group_id=bded54c235c74f12a167629af5a6e94e)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:44.333138 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f12ca6e-62d6-40a2-bc9c-87da3f7af1d8 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:44.334936 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 216/439] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 5 10:13:44 2022] GET /identity/v3/groups/bded54c235c74f12a167629af5a6e94e => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:44.335619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f12ca6e-62d6-40a2-bc9c-87da3f7af1d8 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:44.344413 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1217d2eb-1a8f-4540-b2ff-c8893b01f109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:44.345160 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1217d2eb-1a8f-4540-b2ff-c8893b01f109 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:44.345417 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1217d2eb-1a8f-4540-b2ff-c8893b01f109 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:44.345716 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1217d2eb-1a8f-4540-b2ff-c8893b01f109 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:44.346598 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1217d2eb-1a8f-4540-b2ff-c8893b01f109 None admin] RBAC: Adding query filter params (group.id=bded54c235c74f12a167629af5a6e94e, role.id=c4ba94fa4bd34a09ba87d357142d9db8, scope.project.id=a9d326f0c0fa4a38aa7ba8c8c6797c5a) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:44.346940 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1217d2eb-1a8f-4540-b2ff-c8893b01f109 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:44.348592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1217d2eb-1a8f-4540-b2ff-c8893b01f109 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:44.367298 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 224/440] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Aug 5 10:13:44 2022] GET /identity/v3/role_assignments?group.id=bded54c235c74f12a167629af5a6e94e&scope.project.id=a9d326f0c0fa4a38aa7ba8c8c6797c5a&role.id=c4ba94fa4bd34a09ba87d357142d9db8 => generated 611 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:44.368132 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1217d2eb-1a8f-4540-b2ff-c8893b01f109 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:44.402351 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:44.455675 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f12ca6e-62d6-40a2-bc9c-87da3f7af1d8 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:44.495592 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8c46d2-e13c-4ce3-8c7a-cc433ddc33ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:44.496170 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c46d2-e13c-4ce3-8c7a-cc433ddc33ba None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:44.496444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c46d2-e13c-4ce3-8c7a-cc433ddc33ba None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:44.496709 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c46d2-e13c-4ce3-8c7a-cc433ddc33ba None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:44.506296 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1217d2eb-1a8f-4540-b2ff-c8893b01f109 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:44.509746 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8c46d2-e13c-4ce3-8c7a-cc433ddc33ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:44.515031 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-6d8c46d2-e13c-4ce3-8c7a-cc433ddc33ba None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 10:13:44.515031 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:44.515031 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:44.515031 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:44.515031 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:44.515031 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:44.515031 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:44.515031 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:44.515031 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:44.515031 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:44.515031 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:44.515031 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:44.515031 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:44.515031 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:44.515031 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:44.515031 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:44.515929 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:44.515929 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:44.515929 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:44.515929 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:44.515929 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:44.515929 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:44.515929 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:44.515929 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:44.515929 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:44.515929 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:44.515929 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:44.515929 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:44.515929 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:44.515929 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:44.515929 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:44.515929 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:44.516768 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:44.516768 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:44.516768 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:44.516768 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:44.516768 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:44.516768 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:44.516768 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:44.516768 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:44.516768 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:44.516768 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:44.516768 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 10:13:44.516768 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:13:44.516768 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 217/441] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 5 10:13:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:44.517430 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a669b81-aca5-4154-ac7c-dd13b64f86fa None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:44.518435 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a669b81-aca5-4154-ac7c-dd13b64f86fa None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:44.518435 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a669b81-aca5-4154-ac7c-dd13b64f86fa None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:44.519566 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 218/442] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:44.523799 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a669b81-aca5-4154-ac7c-dd13b64f86fa None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:44.523959 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:44.550650 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1186b753-fde7-4451-8a88-f4002dc49060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:44.551326 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186b753-fde7-4451-8a88-f4002dc49060 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:44.551608 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186b753-fde7-4451-8a88-f4002dc49060 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:44.551890 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186b753-fde7-4451-8a88-f4002dc49060 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:44.552751 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1186b753-fde7-4451-8a88-f4002dc49060 None admin] RBAC: Adding query filter params (group.id=bded54c235c74f12a167629af5a6e94e, role.id=c4ba94fa4bd34a09ba87d357142d9db8, scope.project.id=400a983b4cd947d487b7fa86a5f5c339) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:44.553030 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1186b753-fde7-4451-8a88-f4002dc49060 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:44.554474 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1186b753-fde7-4451-8a88-f4002dc49060 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:44.561263 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 225/443] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Aug 5 10:13:44 2022] GET /identity/v3/role_assignments?group.id=bded54c235c74f12a167629af5a6e94e&scope.project.id=400a983b4cd947d487b7fa86a5f5c339&role.id=c4ba94fa4bd34a09ba87d357142d9db8 => generated 611 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:44.562620 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79da163-96c3-413d-8964-d230e5d33ceb None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:44.562936 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79da163-96c3-413d-8964-d230e5d33ceb None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:44.563207 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79da163-96c3-413d-8964-d230e5d33ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:44.584884 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c79da163-96c3-413d-8964-d230e5d33ceb None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:44.586961 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c79da163-96c3-413d-8964-d230e5d33ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:44.661440 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a669b81-aca5-4154-ac7c-dd13b64f86fa None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:44.682877 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:44.690538 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 226/444] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:44.707598 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c990b16-43f7-494d-9cc9-037a72a70090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:44.708247 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c990b16-43f7-494d-9cc9-037a72a70090 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:44.708528 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c990b16-43f7-494d-9cc9-037a72a70090 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:44.708799 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c990b16-43f7-494d-9cc9-037a72a70090 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:44.709529 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c990b16-43f7-494d-9cc9-037a72a70090 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:44.709834 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c990b16-43f7-494d-9cc9-037a72a70090 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:44.711272 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c990b16-43f7-494d-9cc9-037a72a70090 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:44.718248 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79da163-96c3-413d-8964-d230e5d33ceb None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:44.727475 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 219/445] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:13:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:44.735117 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c990b16-43f7-494d-9cc9-037a72a70090 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:44.848457 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79da163-96c3-413d-8964-d230e5d33ceb None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:44.860682 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:44.863048 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c990b16-43f7-494d-9cc9-037a72a70090 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:44.891320 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d473f09-5323-41d6-ad38-71d2610578a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:44.892357 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d473f09-5323-41d6-ad38-71d2610578a6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:44.892357 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d473f09-5323-41d6-ad38-71d2610578a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:44.892571 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d473f09-5323-41d6-ad38-71d2610578a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:44.905207 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d473f09-5323-41d6-ad38-71d2610578a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:44.906410 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f11860-864c-4bac-9538-541635b0e53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:44.906883 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f11860-864c-4bac-9538-541635b0e53e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:44.907195 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f11860-864c-4bac-9538-541635b0e53e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:44.907470 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f11860-864c-4bac-9538-541635b0e53e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:44.911083 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-2d473f09-5323-41d6-ad38-71d2610578a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:13:44.911083 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:44.911083 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:44.911083 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:44.911083 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:44.911083 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:44.911083 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:44.911083 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:44.911083 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:44.911083 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:44.911083 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:44.911083 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:44.911083 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:44.911083 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:44.911083 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:44.911083 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:44.912168 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:44.912168 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:44.912168 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:44.912168 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:44.912168 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:44.912168 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:44.912168 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:44.912168 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:44.912168 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:44.912168 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:44.912168 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:44.912168 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:44.912168 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:44.912168 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:44.912168 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:44.912168 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:44.913139 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:44.913139 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:44.913139 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:44.913139 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:44.913139 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:44.913139 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:44.913139 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:44.913139 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:44.913139 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:44.913139 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:44.913139 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:13:44.913139 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:44.913139 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 227/446] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 10:13:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:13:44.920495 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d473f09-5323-41d6-ad38-71d2610578a6 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:44.968311 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:44.997318 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f11860-864c-4bac-9538-541635b0e53e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:44.997318 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f11860-864c-4bac-9538-541635b0e53e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:45.042777 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d473f09-5323-41d6-ad38-71d2610578a6 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:45.084665 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbea274-ad13-40b4-aaa7-708fbbcecd0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:45.084906 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbea274-ad13-40b4-aaa7-708fbbcecd0f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:45.085186 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbea274-ad13-40b4-aaa7-708fbbcecd0f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:45.085460 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbea274-ad13-40b4-aaa7-708fbbcecd0f None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:45.086172 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbea274-ad13-40b4-aaa7-708fbbcecd0f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:45.086478 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbea274-ad13-40b4-aaa7-708fbbcecd0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:45.087930 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbea274-ad13-40b4-aaa7-708fbbcecd0f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:45.100507 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 228/447] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:13:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:45.106345 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbea274-ad13-40b4-aaa7-708fbbcecd0f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:45.119468 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 220/448] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:13:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 10:13:45.126178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f11860-864c-4bac-9538-541635b0e53e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:45.140484 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:45.208166 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:45.214857 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbea274-ad13-40b4-aaa7-708fbbcecd0f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:45.227652 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f11860-864c-4bac-9538-541635b0e53e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:45.249572 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f72b4de-d025-4abc-88b3-964e1ab5e7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:45.250538 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f72b4de-d025-4abc-88b3-964e1ab5e7fc None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:45.250812 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f72b4de-d025-4abc-88b3-964e1ab5e7fc None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:45.251138 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f72b4de-d025-4abc-88b3-964e1ab5e7fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:45.260310 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a8ff09-2228-4bab-8102-050723bd2cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:45.260982 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8ff09-2228-4bab-8102-050723bd2cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:45.261275 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8ff09-2228-4bab-8102-050723bd2cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:45.261550 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8ff09-2228-4bab-8102-050723bd2cb1 None admin] PATH_INFO: `/v3/projects/400a983b4cd947d487b7fa86a5f5c339` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:45.262705 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a8ff09-2228-4bab-8102-050723bd2cb1 None admin] RBAC: Authorizing `identity:get_project(project_id=400a983b4cd947d487b7fa86a5f5c339)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:45.264268 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a8ff09-2228-4bab-8102-050723bd2cb1 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:45.266143 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 221/449] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:13:45 2022] GET /identity/v3/projects/400a983b4cd947d487b7fa86a5f5c339 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:45.271835 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a8ff09-2228-4bab-8102-050723bd2cb1 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:45.320101 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f72b4de-d025-4abc-88b3-964e1ab5e7fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:45.321566 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f72b4de-d025-4abc-88b3-964e1ab5e7fc None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:45.349958 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:45.368535 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a8ff09-2228-4bab-8102-050723bd2cb1 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:45.401353 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384fbc0e-d2fe-4a14-bc75-ef6a068ce0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:45.402114 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384fbc0e-d2fe-4a14-bc75-ef6a068ce0d5 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:45.402503 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384fbc0e-d2fe-4a14-bc75-ef6a068ce0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:45.402778 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384fbc0e-d2fe-4a14-bc75-ef6a068ce0d5 None admin] PATH_INFO: `/v3/groups/bded54c235c74f12a167629af5a6e94e` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:45.404421 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384fbc0e-d2fe-4a14-bc75-ef6a068ce0d5 None admin] RBAC: Authorizing `identity:get_group(group_id=bded54c235c74f12a167629af5a6e94e)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:45.405891 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384fbc0e-d2fe-4a14-bc75-ef6a068ce0d5 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:45.407572 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 222/450] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 5 10:13:45 2022] GET /identity/v3/groups/bded54c235c74f12a167629af5a6e94e => generated 226 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:13:45.413400 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384fbc0e-d2fe-4a14-bc75-ef6a068ce0d5 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:45.425067 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 229/451] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:13:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 10:13:45.430722 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f72b4de-d025-4abc-88b3-964e1ab5e7fc None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:45.432427 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:45.512756 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384fbc0e-d2fe-4a14-bc75-ef6a068ce0d5 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:45.526933 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f72b4de-d025-4abc-88b3-964e1ab5e7fc None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:45.548657 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb834fb-d593-4be5-8666-b8f25a94fd3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:45.549276 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb834fb-d593-4be5-8666-b8f25a94fd3a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:45.549560 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb834fb-d593-4be5-8666-b8f25a94fd3a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:45.549832 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb834fb-d593-4be5-8666-b8f25a94fd3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:45.550644 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb834fb-d593-4be5-8666-b8f25a94fd3a None admin] RBAC: Adding query filter params (group.id=bded54c235c74f12a167629af5a6e94e, role.id=57a070b43ad940f1886c69721dd304c2, scope.project.id=400a983b4cd947d487b7fa86a5f5c339) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:45.550989 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb834fb-d593-4be5-8666-b8f25a94fd3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:45.552519 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb834fb-d593-4be5-8666-b8f25a94fd3a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:45.555749 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:45.558453 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 223/452] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Aug 5 10:13:45 2022] GET /identity/v3/role_assignments?group.id=bded54c235c74f12a167629af5a6e94e&scope.project.id=400a983b4cd947d487b7fa86a5f5c339&role.id=57a070b43ad940f1886c69721dd304c2 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:13:45.562554 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87382067-898c-4f97-94df-01c181f57623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:45.563252 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87382067-898c-4f97-94df-01c181f57623 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:45.563509 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87382067-898c-4f97-94df-01c181f57623 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:45.563796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87382067-898c-4f97-94df-01c181f57623 None admin] PATH_INFO: `/v3/projects/be89c43472bc49a2bd0e46cddf175958` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:45.565082 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87382067-898c-4f97-94df-01c181f57623 None admin] RBAC: Authorizing `identity:get_project(project_id=be89c43472bc49a2bd0e46cddf175958)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:45.566565 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87382067-898c-4f97-94df-01c181f57623 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:45.568354 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 230/453] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:13:45 2022] GET /identity/v3/projects/be89c43472bc49a2bd0e46cddf175958 => generated 344 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:13:45.574416 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb834fb-d593-4be5-8666-b8f25a94fd3a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:45.661800 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:45.682599 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb834fb-d593-4be5-8666-b8f25a94fd3a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:45.719960 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d76c2b-c81a-4243-9ace-211bf3ca91b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:45.720620 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d76c2b-c81a-4243-9ace-211bf3ca91b1 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:45.720903 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d76c2b-c81a-4243-9ace-211bf3ca91b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:45.721176 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d76c2b-c81a-4243-9ace-211bf3ca91b1 None admin] PATH_INFO: `/v3/groups/f8ba520a72c14d3f9de2539d0c047a2c` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:45.722353 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d76c2b-c81a-4243-9ace-211bf3ca91b1 None admin] RBAC: Authorizing `identity:get_group(group_id=f8ba520a72c14d3f9de2539d0c047a2c)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:45.723853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d76c2b-c81a-4243-9ace-211bf3ca91b1 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:45.725464 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 224/454] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 5 10:13:45 2022] GET /identity/v3/groups/f8ba520a72c14d3f9de2539d0c047a2c => generated 229 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:45.732658 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87382067-898c-4f97-94df-01c181f57623 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:45.835162 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87382067-898c-4f97-94df-01c181f57623 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:45.870667 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821121ab-c666-42f4-adf6-6551844a4781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:45.871384 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821121ab-c666-42f4-adf6-6551844a4781 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:45.871657 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821121ab-c666-42f4-adf6-6551844a4781 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:45.871927 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821121ab-c666-42f4-adf6-6551844a4781 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:45.872719 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821121ab-c666-42f4-adf6-6551844a4781 None admin] RBAC: Adding query filter params (group.id=f8ba520a72c14d3f9de2539d0c047a2c, role.id=daef89adcaeb48a382a0bcb03340ccbb, scope.project.id=be89c43472bc49a2bd0e46cddf175958) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:45.873029 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821121ab-c666-42f4-adf6-6551844a4781 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:45.874484 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821121ab-c666-42f4-adf6-6551844a4781 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:45.880714 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 231/455] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Aug 5 10:13:45 2022] GET /identity/v3/role_assignments?group.id=f8ba520a72c14d3f9de2539d0c047a2c&scope.project.id=be89c43472bc49a2bd0e46cddf175958&role.id=daef89adcaeb48a382a0bcb03340ccbb => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:48.257079 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36b62e-3e75-4023-94ed-2f1db36fb52c None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:48.257963 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36b62e-3e75-4023-94ed-2f1db36fb52c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:48.258373 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36b62e-3e75-4023-94ed-2f1db36fb52c None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:48.260186 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 225/456] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:48.265586 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a781129-9b2c-4dd5-9bad-74c4e1553809 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:48.266328 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a781129-9b2c-4dd5-9bad-74c4e1553809 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:48.266733 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a781129-9b2c-4dd5-9bad-74c4e1553809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:48.291329 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-5a781129-9b2c-4dd5-9bad-74c4e1553809 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:48.292863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5a781129-9b2c-4dd5-9bad-74c4e1553809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:48.373683 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:48.383903 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 232/457] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:48.417407 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e36b62e-3e75-4023-94ed-2f1db36fb52c None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:48.433996 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9b991-46db-4110-bacf-6674294ef497 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:48.434330 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9b991-46db-4110-bacf-6674294ef497 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:48.434531 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9b991-46db-4110-bacf-6674294ef497 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:48.438468 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 233/458] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:48.442937 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5c71b-c610-409a-830c-fefa10e5a468 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:48.442937 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5c71b-c610-409a-830c-fefa10e5a468 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:48.442937 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5c71b-c610-409a-830c-fefa10e5a468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:48.460652 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-59d5c71b-c610-409a-830c-fefa10e5a468 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:48.462073 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-59d5c71b-c610-409a-830c-fefa10e5a468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:48.531111 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 234/459] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 10:13:48.555544 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d5c71b-c610-409a-830c-fefa10e5a468 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:48.657015 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e36b62e-3e75-4023-94ed-2f1db36fb52c None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:48.662871 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d5c71b-c610-409a-830c-fefa10e5a468 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:48.692860 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:48.702084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3694530d-aee9-4ce5-8204-c8403a960723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:48.703091 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694530d-aee9-4ce5-8204-c8403a960723 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:48.703546 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694530d-aee9-4ce5-8204-c8403a960723 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:48.704010 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694530d-aee9-4ce5-8204-c8403a960723 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:48.715658 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3694530d-aee9-4ce5-8204-c8403a960723 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:48.717692 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3694530d-aee9-4ce5-8204-c8403a960723 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:48.734478 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-3694530d-aee9-4ce5-8204-c8403a960723 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:13:48.734478 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:48.734478 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:48.734478 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:48.734478 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:48.734478 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:48.734478 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:48.734478 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:48.734478 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:48.734478 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:48.734478 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:48.734478 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:48.734478 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:13:48.734478 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:13:48.734478 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:13:48.735426 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:13:48.735426 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:48.735426 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:48.735426 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:48.735426 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:48.735426 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:48.735426 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:48.735426 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:48.735426 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:48.735426 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:48.735426 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:48.735426 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:48.735426 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:48.735426 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:48.735426 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:48.735426 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:48.735426 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:48.736259 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:13:48.736259 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:13:48.736259 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:13:48.736259 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:48.738432 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 235/460] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:13:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:13:48.744386 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3694530d-aee9-4ce5-8204-c8403a960723 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:48.773257 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a136d-a51d-4a13-9b87-bb41073f4f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:48.774226 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a136d-a51d-4a13-9b87-bb41073f4f4a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:48.774708 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a136d-a51d-4a13-9b87-bb41073f4f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:48.775174 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a136d-a51d-4a13-9b87-bb41073f4f4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:48.800606 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8a136d-a51d-4a13-9b87-bb41073f4f4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:48.802420 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8a136d-a51d-4a13-9b87-bb41073f4f4a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:48.824349 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:48.838503 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-9d8a136d-a51d-4a13-9b87-bb41073f4f4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:13:48.838503 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:48.838503 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:48.838503 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:48.838503 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:48.838503 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:48.838503 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:48.838503 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:48.838503 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:48.838503 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:48.838503 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:48.838503 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:48.838503 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:13:48.838503 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:13:48.838503 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:13:48.839455 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:13:48.839455 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:48.839455 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:48.839455 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:48.839455 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:48.839455 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:48.839455 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:48.839455 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:48.839455 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:48.839455 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:48.839455 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:48.839455 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:48.839455 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:48.839455 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:48.839455 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:48.839455 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:48.839455 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:48.840285 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:13:48.840285 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:13:48.840285 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:13:48.840285 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:13:48.844129 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 226/461] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:13:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 427 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:48.849715 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a136d-a51d-4a13-9b87-bb41073f4f4a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:48.909724 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3694530d-aee9-4ce5-8204-c8403a960723 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:48.945616 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1225a6-38fe-488d-923b-c8d1be8fc8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:48.946474 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1225a6-38fe-488d-923b-c8d1be8fc8af None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:48.946942 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1225a6-38fe-488d-923b-c8d1be8fc8af None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:48.947456 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1225a6-38fe-488d-923b-c8d1be8fc8af None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:48.948390 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1225a6-38fe-488d-923b-c8d1be8fc8af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:48.948854 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1225a6-38fe-488d-923b-c8d1be8fc8af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:48.950476 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1225a6-38fe-488d-923b-c8d1be8fc8af None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:48.964057 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 236/462] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:13:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:48.970399 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6834b-5482-4a24-bb54-45d1caf66992 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:48.970966 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6834b-5482-4a24-bb54-45d1caf66992 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:48.971537 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6834b-5482-4a24-bb54-45d1caf66992 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:48.973262 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 237/463] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:48.974149 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6834b-5482-4a24-bb54-45d1caf66992 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:49.007366 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a136d-a51d-4a13-9b87-bb41073f4f4a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:49.011818 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:49.046031 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5044fa6-f0c0-4885-b31f-be6aeb3834a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:49.047069 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5044fa6-f0c0-4885-b31f-be6aeb3834a8 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:49.047481 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5044fa6-f0c0-4885-b31f-be6aeb3834a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:49.047882 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5044fa6-f0c0-4885-b31f-be6aeb3834a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:49.048854 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5044fa6-f0c0-4885-b31f-be6aeb3834a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:49.049163 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5044fa6-f0c0-4885-b31f-be6aeb3834a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:49.050554 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5044fa6-f0c0-4885-b31f-be6aeb3834a8 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:49.063234 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 227/464] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:13:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:13:49.064328 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ac790-f8bd-4e8c-b1ce-269044c40dca None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:49.064611 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ac790-f8bd-4e8c-b1ce-269044c40dca None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:49.064884 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ac790-f8bd-4e8c-b1ce-269044c40dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:49.083794 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ae0ac790-f8bd-4e8c-b1ce-269044c40dca None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:49.085054 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ae0ac790-f8bd-4e8c-b1ce-269044c40dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:49.087585 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6834b-5482-4a24-bb54-45d1caf66992 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:49.124852 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c7ed3-f09e-417a-a629-90c2f492bc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:49.125574 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c7ed3-f09e-417a-a629-90c2f492bc2a None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:49.125774 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c7ed3-f09e-417a-a629-90c2f492bc2a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:49.126013 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c7ed3-f09e-417a-a629-90c2f492bc2a None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:49.126712 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335c7ed3-f09e-417a-a629-90c2f492bc2a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:49.128180 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335c7ed3-f09e-417a-a629-90c2f492bc2a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:49.141987 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:49.151629 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 238/465] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 5 10:13:48 2022] POST /identity/v3/users => generated 248 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:13:49.153244 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9979511-cf2f-4ba8-bc3d-32c288e86546 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:49.153526 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9979511-cf2f-4ba8-bc3d-32c288e86546 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:49.153797 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9979511-cf2f-4ba8-bc3d-32c288e86546 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:49.155498 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 239/466] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:49.156234 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9979511-cf2f-4ba8-bc3d-32c288e86546 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:49.170352 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 228/467] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:49.172308 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c8780b-b72b-42ad-a731-9f60b8353d79 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:49.172819 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c8780b-b72b-42ad-a731-9f60b8353d79 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:49.173287 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c8780b-b72b-42ad-a731-9f60b8353d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:49.201564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-10c8780b-b72b-42ad-a731-9f60b8353d79 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:49.205286 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-10c8780b-b72b-42ad-a731-9f60b8353d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:49.291503 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:49.304586 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 229/468] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:49.305584 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c8780b-b72b-42ad-a731-9f60b8353d79 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:49.308562 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9979511-cf2f-4ba8-bc3d-32c288e86546 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:49.355224 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb8ce8-72cc-41ae-99a3-f972c84646ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:49.356208 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb8ce8-72cc-41ae-99a3-f972c84646ee None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:49.356685 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb8ce8-72cc-41ae-99a3-f972c84646ee None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:49.357193 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb8ce8-72cc-41ae-99a3-f972c84646ee None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:49.358239 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcb8ce8-72cc-41ae-99a3-f972c84646ee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:49.359750 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcb8ce8-72cc-41ae-99a3-f972c84646ee None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:49.383817 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 240/469] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 5 10:13:49 2022] POST /identity/v3/users => generated 246 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:13:49.384544 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb8ce8-72cc-41ae-99a3-f972c84646ee None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:49.431240 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c8780b-b72b-42ad-a731-9f60b8353d79 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:49.458372 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:49.478321 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e74982b-c289-4c08-ab75-a12c2723cd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:49.479029 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74982b-c289-4c08-ab75-a12c2723cd57 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:49.479313 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74982b-c289-4c08-ab75-a12c2723cd57 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:49.479590 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74982b-c289-4c08-ab75-a12c2723cd57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:49.492520 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e74982b-c289-4c08-ab75-a12c2723cd57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:49.493638 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e74982b-c289-4c08-ab75-a12c2723cd57 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:49.513822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-3e74982b-c289-4c08-ab75-a12c2723cd57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:13:49.513822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:49.513822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:49.513822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:49.513822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:49.513822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:49.513822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:49.513822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:49.513822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:49.513822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:49.513822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:49.513822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:49.513822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:13:49.513822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:13:49.513822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:13:49.514827 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:13:49.514827 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:49.514827 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:49.514827 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:49.514827 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:49.514827 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:49.514827 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:49.514827 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:49.514827 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:49.514827 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:49.514827 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:49.514827 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:49.514827 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:49.514827 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:49.514827 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:49.514827 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:49.514827 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:49.515736 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:13:49.515736 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:13:49.515736 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:13:49.515736 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:13:49.515736 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 230/470] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:13:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:49.521596 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e74982b-c289-4c08-ab75-a12c2723cd57 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:49.539186 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb8ce8-72cc-41ae-99a3-f972c84646ee None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:49.589202 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfabaa9a-14fd-440c-87ee-9a7d916efc02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:49.590645 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfabaa9a-14fd-440c-87ee-9a7d916efc02 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:49.591427 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfabaa9a-14fd-440c-87ee-9a7d916efc02 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:49.591887 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfabaa9a-14fd-440c-87ee-9a7d916efc02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:49.607254 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfabaa9a-14fd-440c-87ee-9a7d916efc02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:49.609202 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfabaa9a-14fd-440c-87ee-9a7d916efc02 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:49.617649 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:49.630220 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-bfabaa9a-14fd-440c-87ee-9a7d916efc02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:13:49.630220 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:49.630220 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:49.630220 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:49.630220 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:49.630220 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:49.630220 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:49.630220 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:49.630220 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:49.630220 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:49.630220 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:49.630220 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:49.630220 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:13:49.630220 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:13:49.630220 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:13:49.631148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:13:49.631148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:49.631148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:49.631148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:49.631148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:49.631148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:49.631148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:49.631148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:49.631148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:49.631148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:49.631148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:49.631148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:49.631148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:49.631148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:49.631148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:49.631148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:49.631148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:49.632785 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:13:49.632785 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:13:49.632785 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:13:49.632785 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:49.639082 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 241/471] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:13:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:49.641654 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfabaa9a-14fd-440c-87ee-9a7d916efc02 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:49.658099 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e74982b-c289-4c08-ab75-a12c2723cd57 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:49.704868 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db890d10-336b-4e0f-803c-da26bf7e8ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:49.706037 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db890d10-336b-4e0f-803c-da26bf7e8ca3 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:49.706827 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db890d10-336b-4e0f-803c-da26bf7e8ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:49.707588 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db890d10-336b-4e0f-803c-da26bf7e8ca3 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:49.708559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db890d10-336b-4e0f-803c-da26bf7e8ca3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:49.709121 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db890d10-336b-4e0f-803c-da26bf7e8ca3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:49.711267 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db890d10-336b-4e0f-803c-da26bf7e8ca3 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:49.726971 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 231/472] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:13:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:49.736163 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db890d10-336b-4e0f-803c-da26bf7e8ca3 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:49.757270 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfabaa9a-14fd-440c-87ee-9a7d916efc02 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:49.775936 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:49.799654 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dc5062-b65e-46fd-a1d2-4fc0f6dba3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:49.800297 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc5062-b65e-46fd-a1d2-4fc0f6dba3ba None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:49.800581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc5062-b65e-46fd-a1d2-4fc0f6dba3ba None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:49.800859 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc5062-b65e-46fd-a1d2-4fc0f6dba3ba None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:49.801564 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dc5062-b65e-46fd-a1d2-4fc0f6dba3ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:49.801866 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dc5062-b65e-46fd-a1d2-4fc0f6dba3ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:49.803301 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dc5062-b65e-46fd-a1d2-4fc0f6dba3ba None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:49.818181 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 242/473] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:13:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:49.819548 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f3666-e017-4893-94dc-65496bf04581 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:49.819835 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f3666-e017-4893-94dc-65496bf04581 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:49.820108 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f3666-e017-4893-94dc-65496bf04581 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:49.821569 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 243/474] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:49.824671 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f3666-e017-4893-94dc-65496bf04581 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:49.861235 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db890d10-336b-4e0f-803c-da26bf7e8ca3 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:49.893605 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:49.904715 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb567fef-7fe5-4854-8a6c-638326b39e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:49.906054 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb567fef-7fe5-4854-8a6c-638326b39e11 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:49.906340 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb567fef-7fe5-4854-8a6c-638326b39e11 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:49.906746 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb567fef-7fe5-4854-8a6c-638326b39e11 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:49.907606 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb567fef-7fe5-4854-8a6c-638326b39e11 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:49.910206 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb567fef-7fe5-4854-8a6c-638326b39e11 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:49.940168 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 232/475] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 5 10:13:49 2022] POST /identity/v3/users => generated 249 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:13:49.941758 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab6c61-1192-414c-8f95-b8faa6057458 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:49.942194 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab6c61-1192-414c-8f95-b8faa6057458 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:49.942582 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab6c61-1192-414c-8f95-b8faa6057458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:49.943714 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f3666-e017-4893-94dc-65496bf04581 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:49.967074 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-5bab6c61-1192-414c-8f95-b8faa6057458 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:49.968502 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5bab6c61-1192-414c-8f95-b8faa6057458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:49.987423 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a937b936-71ae-455b-b28b-6050600371ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:49.988149 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a937b936-71ae-455b-b28b-6050600371ed None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:49.988434 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a937b936-71ae-455b-b28b-6050600371ed None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:49.988808 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a937b936-71ae-455b-b28b-6050600371ed None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:49.989562 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a937b936-71ae-455b-b28b-6050600371ed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:49.991411 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a937b936-71ae-455b-b28b-6050600371ed None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:50.026243 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 244/476] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 5 10:13:49 2022] POST /identity/v3/users => generated 248 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:13:50.071017 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 233/477] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:50.096292 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a937b936-71ae-455b-b28b-6050600371ed None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:50.255798 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a937b936-71ae-455b-b28b-6050600371ed None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:50.270391 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:50.295191 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e367be3e-80b3-4092-ad7d-b22ea5bf709a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:50.295817 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367be3e-80b3-4092-ad7d-b22ea5bf709a None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:50.296101 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367be3e-80b3-4092-ad7d-b22ea5bf709a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:50.296376 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367be3e-80b3-4092-ad7d-b22ea5bf709a None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:50.297096 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e367be3e-80b3-4092-ad7d-b22ea5bf709a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:50.298497 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e367be3e-80b3-4092-ad7d-b22ea5bf709a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:50.309471 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.sql.core [None req-e367be3e-80b3-4092-ad7d-b22ea5bf709a None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 05 10:13:50.309471 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 05 10:13:50.309471 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [parameters: {'id': 'c5457d1cb559441ebd9f77635bceb0b7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 05 10:13:50.309471 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80142) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 05 10:13:50.327273 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-e367be3e-80b3-4092-ad7d-b22ea5bf709a None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 05 10:13:50.327273 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:50.327273 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 05 10:13:50.327273 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 05 10:13:50.327273 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 05 10:13:50.327273 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 05 10:13:50.327273 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 05 10:13:50.327273 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application result = self._query(query) Aug 05 10:13:50.327273 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 05 10:13:50.327273 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application conn.query(q) Aug 05 10:13:50.327273 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 05 10:13:50.327273 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 05 10:13:50.327273 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 05 10:13:50.327273 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application result.read() Aug 05 10:13:50.327273 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 05 10:13:50.328175 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 05 10:13:50.328175 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 05 10:13:50.328175 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 05 10:13:50.328175 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 05 10:13:50.328175 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 05 10:13:50.328175 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 05 10:13:50.328175 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 05 10:13:50.328175 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 05 10:13:50.328175 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:50.328175 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 05 10:13:50.328175 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:50.328175 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:50.328175 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 05 10:13:50.328175 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 05 10:13:50.328175 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 05 10:13:50.328175 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return ref.to_dict() Aug 05 10:13:50.328175 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 05 10:13:50.328175 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application next(self.gen) Aug 05 10:13:50.329092 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 05 10:13:50.329092 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application yield resource Aug 05 10:13:50.329092 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 05 10:13:50.329092 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application next(self.gen) Aug 05 10:13:50.329092 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 05 10:13:50.329092 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self.session.rollback() Aug 05 10:13:50.329092 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 05 10:13:50.329092 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self.force_reraise() Aug 05 10:13:50.329092 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 05 10:13:50.329092 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise self.value Aug 05 10:13:50.329092 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 05 10:13:50.329092 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 05 10:13:50.329092 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 05 10:13:50.329092 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application session.commit() Aug 05 10:13:50.329092 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 05 10:13:50.329092 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 05 10:13:50.329092 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 05 10:13:50.329972 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._prepare_impl() Aug 05 10:13:50.329972 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 05 10:13:50.329972 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self.session.flush() Aug 05 10:13:50.329972 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 05 10:13:50.329972 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._flush(objects) Aug 05 10:13:50.329972 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 05 10:13:50.329972 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 05 10:13:50.329972 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 05 10:13:50.329972 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application compat.raise_( Aug 05 10:13:50.329972 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 05 10:13:50.329972 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception Aug 05 10:13:50.329972 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 05 10:13:50.329972 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application flush_context.execute() Aug 05 10:13:50.329972 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 05 10:13:50.329972 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rec.execute(self) Aug 05 10:13:50.329972 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 05 10:13:50.329972 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 05 10:13:50.330863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 05 10:13:50.330863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 05 10:13:50.330863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 05 10:13:50.330863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 05 10:13:50.330863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 05 10:13:50.330863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 05 10:13:50.330863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 05 10:13:50.330863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 05 10:13:50.330863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 05 10:13:50.330863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 05 10:13:50.330863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 05 10:13:50.330863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 05 10:13:50.330863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 05 10:13:50.330863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 05 10:13:50.330863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 05 10:13:50.330863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception Aug 05 10:13:50.332794 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 05 10:13:50.332794 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 05 10:13:50.332794 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 05 10:13:50.332794 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 05 10:13:50.332794 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 05 10:13:50.332794 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application result = self._query(query) Aug 05 10:13:50.332794 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 05 10:13:50.332794 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application conn.query(q) Aug 05 10:13:50.332794 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 05 10:13:50.332794 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 05 10:13:50.332794 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 05 10:13:50.332794 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application result.read() Aug 05 10:13:50.332794 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 05 10:13:50.332794 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 05 10:13:50.332794 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 05 10:13:50.332794 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 05 10:13:50.334714 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 05 10:13:50.334714 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 05 10:13:50.334714 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 05 10:13:50.334714 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 05 10:13:50.334714 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 05 10:13:50.334714 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 05 10:13:50.334714 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [parameters: {'id': 'c5457d1cb559441ebd9f77635bceb0b7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 05 10:13:50.334714 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 05 10:13:50.334714 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:50.334714 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 10:13:50.334714 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:50.334714 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:50.334714 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:50.334714 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:50.334714 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:50.334714 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:50.341630 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:50.341630 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:50.341630 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:50.341630 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:50.341630 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:50.341630 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:50.341630 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 05 10:13:50.341630 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 05 10:13:50.341630 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:50.341630 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:50.341630 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 05 10:13:50.341630 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 05 10:13:50.341630 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 05 10:13:50.341630 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 05 10:13:50.341630 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 05 10:13:50.341630 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:50.343756 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 245/478] 10.222.0.40 () {60 vars in 1131 bytes} [Fri Aug 5 10:13:50 2022] POST /identity/v3/roles => generated 144 bytes in 233 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:50.343893 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bab6c61-1192-414c-8f95-b8faa6057458 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:50.355962 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:50.442599 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bab6c61-1192-414c-8f95-b8faa6057458 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:50.482240 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f1464c-8c5a-4422-b03e-3724c1698a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:50.487109 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1464c-8c5a-4422-b03e-3724c1698a79 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:50.487412 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1464c-8c5a-4422-b03e-3724c1698a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:50.487695 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1464c-8c5a-4422-b03e-3724c1698a79 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:50.499333 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f1464c-8c5a-4422-b03e-3724c1698a79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:50.504449 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-e4f1464c-8c5a-4422-b03e-3724c1698a79 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 10:13:50.504449 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:50.504449 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:50.504449 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:50.504449 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:50.504449 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:50.504449 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:50.504449 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:50.504449 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:50.504449 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:50.504449 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:50.504449 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:50.504449 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:50.504449 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:50.504449 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:50.504449 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:50.506527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:50.506527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:50.506527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:50.506527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:50.506527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:50.506527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:50.506527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:50.506527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:50.506527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:50.506527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:50.506527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:50.506527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:50.506527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:50.506527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:50.506527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:50.506527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:50.508501 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:50.508501 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:50.508501 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:50.508501 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:50.508501 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:50.508501 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:50.508501 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:50.508501 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:50.508501 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:50.508501 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:50.508501 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 10:13:50.508501 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:13:50.508501 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 234/479] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 5 10:13:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:13:50.512596 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e367be3e-80b3-4092-ad7d-b22ea5bf709a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:50.643768 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e367be3e-80b3-4092-ad7d-b22ea5bf709a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:50.690797 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6de077b-546d-418f-8b83-f3bca1d5ed14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:50.691497 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6de077b-546d-418f-8b83-f3bca1d5ed14 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:50.691776 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6de077b-546d-418f-8b83-f3bca1d5ed14 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:50.692050 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6de077b-546d-418f-8b83-f3bca1d5ed14 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:50.692769 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6de077b-546d-418f-8b83-f3bca1d5ed14 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:50.693072 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6de077b-546d-418f-8b83-f3bca1d5ed14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:50.694525 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6de077b-546d-418f-8b83-f3bca1d5ed14 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:50.703507 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 246/480] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:13:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:51.581143 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27246d60-4a5c-46f5-ad0a-0f69b7a950c2 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:51.581143 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27246d60-4a5c-46f5-ad0a-0f69b7a950c2 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:51.581143 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27246d60-4a5c-46f5-ad0a-0f69b7a950c2 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:51.582426 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 235/481] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:51.590962 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c59eb8-02ff-478d-8388-6053fe2c0da1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:51.590962 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c59eb8-02ff-478d-8388-6053fe2c0da1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:51.590962 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c59eb8-02ff-478d-8388-6053fe2c0da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:51.609575 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-46c59eb8-02ff-478d-8388-6053fe2c0da1 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:51.611164 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-46c59eb8-02ff-478d-8388-6053fe2c0da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:51.666435 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:51.715318 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 247/482] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:51.752347 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27246d60-4a5c-46f5-ad0a-0f69b7a950c2 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:51.861321 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:51.889946 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27246d60-4a5c-46f5-ad0a-0f69b7a950c2 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:51.935941 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a4fc5-cedc-4663-9595-ce42c6b68021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:51.936590 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a4fc5-cedc-4663-9595-ce42c6b68021 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:51.936875 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a4fc5-cedc-4663-9595-ce42c6b68021 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:51.937145 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a4fc5-cedc-4663-9595-ce42c6b68021 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:51.952925 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075a4fc5-cedc-4663-9595-ce42c6b68021 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:51.956863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ff022-d856-40ef-a431-6b3bcbf72077 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:51.957370 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ff022-d856-40ef-a431-6b3bcbf72077 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:51.957843 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ff022-d856-40ef-a431-6b3bcbf72077 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:51.958246 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-075a4fc5-cedc-4663-9595-ce42c6b68021 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 10:13:51.958246 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:51.958246 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:51.958246 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:51.958246 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:51.958246 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:51.958246 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:51.958246 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:51.958246 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:51.958246 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:51.958246 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:51.958246 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:51.958246 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:51.958246 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:51.958246 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:51.958246 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:51.959412 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:51.959412 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:51.959412 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:51.959412 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:51.959412 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:51.959412 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:51.959412 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:51.959412 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:51.959412 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:51.959412 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:51.959412 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:51.959412 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:51.959412 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:51.959412 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:51.959412 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:51.959412 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:51.960219 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:51.960219 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:51.960219 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:51.960219 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:51.960219 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:51.960219 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:51.960219 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:51.960219 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:51.960219 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:51.960219 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:51.960219 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 10:13:51.960219 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:13:51.960219 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 236/483] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 10:13:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:51.962662 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 248/484] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 10:13:51.967483 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84ff022-d856-40ef-a431-6b3bcbf72077 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:51.967879 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde349e-0a32-488c-a7f3-ea5384731683 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:51.968170 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde349e-0a32-488c-a7f3-ea5384731683 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:51.968451 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde349e-0a32-488c-a7f3-ea5384731683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:51.992436 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-9fde349e-0a32-488c-a7f3-ea5384731683 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:51.993638 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9fde349e-0a32-488c-a7f3-ea5384731683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:52.097479 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84ff022-d856-40ef-a431-6b3bcbf72077 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:52.137541 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d3a775-8bc0-4ed7-8e31-179ad53c0ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:52.138660 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3a775-8bc0-4ed7-8e31-179ad53c0ba1 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:52.139026 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3a775-8bc0-4ed7-8e31-179ad53c0ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:52.139301 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3a775-8bc0-4ed7-8e31-179ad53c0ba1 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:52.140069 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d3a775-8bc0-4ed7-8e31-179ad53c0ba1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:52.140370 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d3a775-8bc0-4ed7-8e31-179ad53c0ba1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:52.141760 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d3a775-8bc0-4ed7-8e31-179ad53c0ba1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:52.151400 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 249/485] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 5 10:13:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:52.157308 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d3a775-8bc0-4ed7-8e31-179ad53c0ba1 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:52.175886 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 237/486] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:52.194737 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:52.211271 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fde349e-0a32-488c-a7f3-ea5384731683 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:52.272320 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d3a775-8bc0-4ed7-8e31-179ad53c0ba1 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:52.308679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef93b842-fcef-48cc-a303-106ba29ae4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:52.309411 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef93b842-fcef-48cc-a303-106ba29ae4ad None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:52.309687 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef93b842-fcef-48cc-a303-106ba29ae4ad None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:52.310133 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef93b842-fcef-48cc-a303-106ba29ae4ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:52.321236 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef93b842-fcef-48cc-a303-106ba29ae4ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:52.322741 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef93b842-fcef-48cc-a303-106ba29ae4ad None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:52.338376 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-ef93b842-fcef-48cc-a303-106ba29ae4ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:13:52.338376 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:52.338376 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:52.338376 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:52.338376 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:52.338376 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:52.338376 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:52.338376 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:52.338376 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:52.338376 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:52.338376 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:52.338376 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:52.338376 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:13:52.338376 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:13:52.338376 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:13:52.339297 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:13:52.339297 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:52.339297 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:52.339297 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:52.339297 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:52.339297 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:52.339297 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:52.339297 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:52.339297 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:52.339297 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:52.339297 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:52.339297 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:52.339297 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:52.339297 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:52.339297 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:52.339297 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:52.339297 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:52.340135 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:13:52.340135 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:13:52.340135 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:13:52.340135 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:52.343720 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 250/487] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:13:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:52.344101 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:52.346425 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef93b842-fcef-48cc-a303-106ba29ae4ad None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:52.391262 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fde349e-0a32-488c-a7f3-ea5384731683 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:52.438741 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7306719a-4795-4667-9f42-e3755e962968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:52.439438 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306719a-4795-4667-9f42-e3755e962968 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:52.439723 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306719a-4795-4667-9f42-e3755e962968 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:52.439996 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306719a-4795-4667-9f42-e3755e962968 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:52.440757 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:52.458216 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef93b842-fcef-48cc-a303-106ba29ae4ad None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:52.460155 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7306719a-4795-4667-9f42-e3755e962968 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:52.465081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-7306719a-4795-4667-9f42-e3755e962968 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 10:13:52.465081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:52.465081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:52.465081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:52.465081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:52.465081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:52.465081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:52.465081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:52.465081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:52.465081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:52.465081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:52.465081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:52.465081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:52.465081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:52.465081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:52.465081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:52.467236 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:52.467236 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:52.467236 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:52.467236 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:52.467236 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:52.467236 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:52.467236 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:52.467236 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:52.467236 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:52.467236 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:52.467236 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:52.467236 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:52.467236 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:52.467236 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:52.467236 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:52.467236 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:52.472363 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:52.472363 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:52.472363 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:52.472363 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:52.472363 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:52.472363 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:52.472363 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:52.472363 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:52.472363 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:52.472363 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:52.472363 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 10:13:52.472363 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:13:52.475078 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 238/488] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 10:13:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:13:52.481264 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7306719a-4795-4667-9f42-e3755e962968 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:52.508813 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f2ef5-7658-4289-92dc-1264f3f9b345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:52.509443 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f2ef5-7658-4289-92dc-1264f3f9b345 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:52.509713 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f2ef5-7658-4289-92dc-1264f3f9b345 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:52.509976 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f2ef5-7658-4289-92dc-1264f3f9b345 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:52.510669 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2f2ef5-7658-4289-92dc-1264f3f9b345 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:52.511242 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2f2ef5-7658-4289-92dc-1264f3f9b345 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:52.512620 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2f2ef5-7658-4289-92dc-1264f3f9b345 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:52.537189 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 251/489] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:13:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:52.538661 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b945bb-f415-4244-8192-7f3cb4b1b722 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:52.538965 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b945bb-f415-4244-8192-7f3cb4b1b722 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:52.539242 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b945bb-f415-4244-8192-7f3cb4b1b722 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:52.545308 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 252/490] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:52.545308 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b945bb-f415-4244-8192-7f3cb4b1b722 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:52.626645 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7306719a-4795-4667-9f42-e3755e962968 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:52.655430 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b945bb-f415-4244-8192-7f3cb4b1b722 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:52.673043 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:52.685477 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4051daff-dc50-4bc7-85ee-16bc60eddd44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:52.686997 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4051daff-dc50-4bc7-85ee-16bc60eddd44 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:52.687954 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4051daff-dc50-4bc7-85ee-16bc60eddd44 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:52.688413 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4051daff-dc50-4bc7-85ee-16bc60eddd44 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:52.692085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4051daff-dc50-4bc7-85ee-16bc60eddd44 None admin] RBAC: Adding query filter params (name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:52.693796 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4051daff-dc50-4bc7-85ee-16bc60eddd44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:52.693905 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4051daff-dc50-4bc7-85ee-16bc60eddd44 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:52.699400 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120af230-6544-4449-b8cf-3366720fc34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:52.700370 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120af230-6544-4449-b8cf-3366720fc34e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:52.702206 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120af230-6544-4449-b8cf-3366720fc34e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:52.702844 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120af230-6544-4449-b8cf-3366720fc34e None admin] PATH_INFO: `/v3/users/glance` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:52.709306 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 239/491] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 5 10:13:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:52.709306 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de2191-1006-4b51-aa74-6a960b252ba8 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:52.709306 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de2191-1006-4b51-aa74-6a960b252ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:52.709306 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de2191-1006-4b51-aa74-6a960b252ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:52.717698 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120af230-6544-4449-b8cf-3366720fc34e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:52.719514 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120af230-6544-4449-b8cf-3366720fc34e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:52.726722 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-36de2191-1006-4b51-aa74-6a960b252ba8 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:52.728069 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-36de2191-1006-4b51-aa74-6a960b252ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:52.736158 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-120af230-6544-4449-b8cf-3366720fc34e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 05 10:13:52.736158 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:52.736158 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:52.736158 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:52.736158 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:52.736158 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:52.736158 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:52.736158 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:52.736158 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:52.736158 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:52.736158 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:52.736158 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:52.736158 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 10:13:52.736158 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 10:13:52.736158 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 10:13:52.736158 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 10:13:52.737263 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:52.737263 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:52.737263 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 10:13:52.737263 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:13:52.737263 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 10:13:52.737263 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:13:52.737263 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:52.737263 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:52.737263 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:52.737263 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:52.737263 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:52.737263 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:52.737263 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:52.737263 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:52.737263 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:52.737263 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:52.737263 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:52.737263 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:52.738381 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:52.738381 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:52.738381 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 10:13:52.738381 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 10:13:52.738381 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 10:13:52.738381 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 10:13:52.738381 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 10:13:52.738381 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 10:13:52.738381 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 05 10:13:52.738381 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:52.741068 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 253/492] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:13:52 2022] GET /identity/v3/users/glance => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:13:52.742057 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120af230-6544-4449-b8cf-3366720fc34e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:52.746284 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:52.812873 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 240/493] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:52.813619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36de2191-1006-4b51-aa74-6a960b252ba8 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:52.873795 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120af230-6544-4449-b8cf-3366720fc34e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:52.906286 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ca505-0d9a-4cd6-b1c0-c504393015e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:52.906862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ca505-0d9a-4cd6-b1c0-c504393015e4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:52.907162 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ca505-0d9a-4cd6-b1c0-c504393015e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:52.907430 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ca505-0d9a-4cd6-b1c0-c504393015e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:52.919907 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122ca505-0d9a-4cd6-b1c0-c504393015e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:52.921140 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122ca505-0d9a-4cd6-b1c0-c504393015e4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:52.938319 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36de2191-1006-4b51-aa74-6a960b252ba8 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:52.939329 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-122ca505-0d9a-4cd6-b1c0-c504393015e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:13:52.939329 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:52.939329 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:52.939329 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:52.939329 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:52.939329 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:52.939329 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:52.939329 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:52.939329 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:52.939329 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:52.939329 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:52.939329 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:52.939329 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:13:52.939329 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:13:52.939329 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:13:52.940133 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:13:52.940133 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:52.940133 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:52.940133 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:52.940133 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:52.940133 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:52.940133 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:52.940133 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:52.940133 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:52.940133 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:52.940133 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:52.940133 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:52.940133 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:52.940133 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:52.940133 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:52.940133 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:52.940133 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:52.940996 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:13:52.940996 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:13:52.940996 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:13:52.940996 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:52.942570 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 254/494] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:13:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:52.944006 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904748f-8019-47be-8fd4-724fa9cbc94b None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:52.944478 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904748f-8019-47be-8fd4-724fa9cbc94b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:52.944958 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904748f-8019-47be-8fd4-724fa9cbc94b None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:52.946677 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 255/495] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:52.947588 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1904748f-8019-47be-8fd4-724fa9cbc94b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:52.960891 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:52.977162 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d3f92-1346-436c-b7de-960906f0b66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:52.977893 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d3f92-1346-436c-b7de-960906f0b66f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:52.978171 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d3f92-1346-436c-b7de-960906f0b66f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:52.978587 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d3f92-1346-436c-b7de-960906f0b66f None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:52.979414 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468d3f92-1346-436c-b7de-960906f0b66f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:52.979681 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468d3f92-1346-436c-b7de-960906f0b66f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:52.981329 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468d3f92-1346-436c-b7de-960906f0b66f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:53.006938 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:53.029357 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 241/496] 10.222.0.40 () {56 vars in 1137 bytes} [Fri Aug 5 10:13:52 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:53.029357 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d3f92-1346-436c-b7de-960906f0b66f None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:53.058121 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1904748f-8019-47be-8fd4-724fa9cbc94b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:53.096497 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b944c0a-59f4-46c1-9695-a01994011ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:53.097153 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b944c0a-59f4-46c1-9695-a01994011ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:53.097435 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b944c0a-59f4-46c1-9695-a01994011ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:53.097738 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b944c0a-59f4-46c1-9695-a01994011ba3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:53.108414 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b944c0a-59f4-46c1-9695-a01994011ba3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:53.113836 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-7b944c0a-59f4-46c1-9695-a01994011ba3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 10:13:53.113836 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:53.113836 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:53.113836 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:53.113836 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:53.113836 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:53.113836 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:53.113836 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:53.113836 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:53.113836 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:53.113836 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:53.113836 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:53.113836 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:53.113836 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:53.113836 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:53.113836 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:53.114967 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:53.114967 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:53.114967 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:53.114967 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:53.114967 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:53.114967 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:53.114967 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:53.114967 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:53.114967 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:53.114967 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:53.114967 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:53.114967 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:53.114967 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:53.114967 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:53.114967 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:53.114967 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:53.120958 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:53.120958 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:53.120958 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:53.120958 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:53.120958 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:53.120958 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:53.120958 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:53.120958 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:53.120958 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:53.120958 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:53.120958 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 10:13:53.120958 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:53.120958 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 256/497] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 10:13:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:53.120958 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838eca23-8e75-49e9-b99d-07b68d993ba1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:53.121677 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838eca23-8e75-49e9-b99d-07b68d993ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:53.121677 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838eca23-8e75-49e9-b99d-07b68d993ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:53.135346 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-838eca23-8e75-49e9-b99d-07b68d993ba1 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:53.136527 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-838eca23-8e75-49e9-b99d-07b68d993ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:53.143886 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d3f92-1346-436c-b7de-960906f0b66f None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:53.184122 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7591bc34-004d-4d95-aafe-6bdc67ef2eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:53.184747 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591bc34-004d-4d95-aafe-6bdc67ef2eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:53.185061 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591bc34-004d-4d95-aafe-6bdc67ef2eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:53.185337 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591bc34-004d-4d95-aafe-6bdc67ef2eb8 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:53.186173 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7591bc34-004d-4d95-aafe-6bdc67ef2eb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:53.186482 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7591bc34-004d-4d95-aafe-6bdc67ef2eb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:53.188024 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7591bc34-004d-4d95-aafe-6bdc67ef2eb8 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:53.199949 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 242/498] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:13:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:53.205058 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7591bc34-004d-4d95-aafe-6bdc67ef2eb8 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:53.218936 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 257/499] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:53.218936 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838eca23-8e75-49e9-b99d-07b68d993ba1 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:53.248310 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:53.275524 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:53.311313 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7591bc34-004d-4d95-aafe-6bdc67ef2eb8 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:53.333101 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838eca23-8e75-49e9-b99d-07b68d993ba1 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:53.346022 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa09870-241c-41b7-a2a2-e38d42d3c7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:53.346643 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa09870-241c-41b7-a2a2-e38d42d3c7a5 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:53.346942 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa09870-241c-41b7-a2a2-e38d42d3c7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:53.347213 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa09870-241c-41b7-a2a2-e38d42d3c7a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:53.369571 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ebff8-522d-447b-a3ab-457d8fd7f442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:53.370333 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ebff8-522d-447b-a3ab-457d8fd7f442 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:53.370619 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ebff8-522d-447b-a3ab-457d8fd7f442 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:53.370892 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ebff8-522d-447b-a3ab-457d8fd7f442 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:53.371843 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0ebff8-522d-447b-a3ab-457d8fd7f442 None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:53.372157 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0ebff8-522d-447b-a3ab-457d8fd7f442 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:53.373596 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0ebff8-522d-447b-a3ab-457d8fd7f442 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:53.387398 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 258/500] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 5 10:13:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:53.388471 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ebff8-522d-447b-a3ab-457d8fd7f442 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:53.415384 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa09870-241c-41b7-a2a2-e38d42d3c7a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:53.416766 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa09870-241c-41b7-a2a2-e38d42d3c7a5 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:53.455113 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:53.493607 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ebff8-522d-447b-a3ab-457d8fd7f442 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:53.502649 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:53.523127 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 243/501] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:13:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 10:13:53.523868 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa09870-241c-41b7-a2a2-e38d42d3c7a5 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:53.535299 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e14e8-3e4e-4b45-a48b-abbacf1f0efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:53.536064 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e14e8-3e4e-4b45-a48b-abbacf1f0efc None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:53.536344 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e14e8-3e4e-4b45-a48b-abbacf1f0efc None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:53.536611 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e14e8-3e4e-4b45-a48b-abbacf1f0efc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:53.544920 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846e14e8-3e4e-4b45-a48b-abbacf1f0efc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:53.546367 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846e14e8-3e4e-4b45-a48b-abbacf1f0efc None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:53.557697 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-846e14e8-3e4e-4b45-a48b-abbacf1f0efc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 10:13:53.557697 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:53.557697 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:53.557697 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:53.557697 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:53.557697 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:53.557697 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:53.557697 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:53.557697 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:53.557697 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:53.557697 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:53.557697 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:53.557697 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 10:13:53.557697 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 10:13:53.557697 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 10:13:53.557697 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 10:13:53.558608 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:53.558608 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:53.558608 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 10:13:53.558608 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:13:53.558608 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 10:13:53.558608 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:13:53.558608 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:53.558608 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:53.558608 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:53.558608 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:53.558608 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:53.558608 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:53.558608 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:53.558608 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:53.558608 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:53.558608 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:53.558608 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:53.558608 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:53.559548 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:53.559548 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:53.559548 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 10:13:53.559548 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 10:13:53.559548 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 10:13:53.559548 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 10:13:53.559548 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 10:13:53.559548 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 10:13:53.559548 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 10:13:53.559548 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:53.559548 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 259/502] 10.222.0.40 () {56 vars in 1093 bytes} [Fri Aug 5 10:13:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:53.560522 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e14e8-3e4e-4b45-a48b-abbacf1f0efc None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:53.620920 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa09870-241c-41b7-a2a2-e38d42d3c7a5 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:53.655313 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f2bde0-e53a-4a82-96f0-7bc97bd9e5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:53.655928 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f2bde0-e53a-4a82-96f0-7bc97bd9e5e4 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:53.656225 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f2bde0-e53a-4a82-96f0-7bc97bd9e5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:53.656494 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f2bde0-e53a-4a82-96f0-7bc97bd9e5e4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:53.661322 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e14e8-3e4e-4b45-a48b-abbacf1f0efc None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:53.663266 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:53.667318 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f2bde0-e53a-4a82-96f0-7bc97bd9e5e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:53.673241 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-38f2bde0-e53a-4a82-96f0-7bc97bd9e5e4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 10:13:53.673241 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:53.673241 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:53.673241 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:53.673241 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:53.673241 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:53.673241 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:53.673241 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:53.673241 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:53.673241 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:53.673241 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:53.673241 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:53.673241 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:53.673241 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:53.673241 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:53.673241 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:53.674383 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:53.674383 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:53.674383 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:53.674383 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:53.674383 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:53.674383 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:53.674383 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:53.674383 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:53.674383 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:53.674383 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:53.674383 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:53.674383 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:53.674383 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:53.674383 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:53.674383 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:53.674383 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:53.675200 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:53.675200 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:53.675200 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:53.675200 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:53.675200 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:53.675200 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:53.675200 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:53.675200 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:53.675200 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:53.675200 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:53.675200 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 10:13:53.675200 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:13:53.675200 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 244/503] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 10:13:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:53.675200 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f2bde0-e53a-4a82-96f0-7bc97bd9e5e4 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:53.700726 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c10f8-dec6-4e05-9b10-c61ff6508a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:53.701374 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c10f8-dec6-4e05-9b10-c61ff6508a28 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:53.701654 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c10f8-dec6-4e05-9b10-c61ff6508a28 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:53.701927 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c10f8-dec6-4e05-9b10-c61ff6508a28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:53.713188 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3c10f8-dec6-4e05-9b10-c61ff6508a28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:53.714648 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3c10f8-dec6-4e05-9b10-c61ff6508a28 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:53.728780 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-da3c10f8-dec6-4e05-9b10-c61ff6508a28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:13:53.728780 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:53.728780 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:53.728780 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:53.728780 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:53.728780 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:53.728780 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:53.728780 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:53.728780 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:53.728780 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:53.728780 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:53.728780 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:53.728780 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:13:53.728780 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:13:53.728780 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:13:53.729649 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:13:53.729649 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:53.729649 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:53.729649 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:53.729649 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:53.729649 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:53.729649 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:53.729649 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:53.729649 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:53.729649 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:53.729649 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:53.729649 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:53.729649 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:53.729649 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:53.729649 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:53.729649 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:53.729649 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:53.730470 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:13:53.730470 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:13:53.730470 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:13:53.730470 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:53.730470 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 260/504] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:13:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:53.731055 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f670356f-128f-4ad5-abc4-69b339216300 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:53.731336 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f670356f-128f-4ad5-abc4-69b339216300 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:53.731640 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f670356f-128f-4ad5-abc4-69b339216300 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:53.733182 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 261/505] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:53.733885 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f670356f-128f-4ad5-abc4-69b339216300 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:53.756107 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:53.776946 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f2bde0-e53a-4a82-96f0-7bc97bd9e5e4 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:53.810386 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b0103-0239-483b-b050-15b34c46137c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:53.811064 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b0103-0239-483b-b050-15b34c46137c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:53.811348 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b0103-0239-483b-b050-15b34c46137c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:53.811619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b0103-0239-483b-b050-15b34c46137c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:53.821628 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033b0103-0239-483b-b050-15b34c46137c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:53.823119 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033b0103-0239-483b-b050-15b34c46137c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:53.837957 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-033b0103-0239-483b-b050-15b34c46137c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:13:53.837957 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:53.837957 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:53.837957 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:53.837957 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:53.837957 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:53.837957 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:53.837957 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:53.837957 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:53.837957 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:53.837957 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:53.837957 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:53.837957 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:13:53.837957 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:13:53.837957 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:13:53.838835 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:13:53.838835 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:53.838835 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:53.838835 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:53.838835 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:53.838835 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:53.838835 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:53.838835 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:53.838835 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:53.838835 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:53.838835 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:53.838835 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:53.838835 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:53.838835 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:53.838835 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:53.838835 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:53.838835 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:53.839702 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:13:53.839702 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:13:53.839702 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:13:53.839702 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:13:53.839903 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f670356f-128f-4ad5-abc4-69b339216300 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:53.839972 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 245/506] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:13:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:53.839972 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b0103-0239-483b-b050-15b34c46137c None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:53.873290 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777944f0-9337-4e76-ac4c-b3fb1a42db41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:53.873898 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777944f0-9337-4e76-ac4c-b3fb1a42db41 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:53.874173 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777944f0-9337-4e76-ac4c-b3fb1a42db41 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:53.874443 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777944f0-9337-4e76-ac4c-b3fb1a42db41 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:53.875265 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777944f0-9337-4e76-ac4c-b3fb1a42db41 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:53.875555 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777944f0-9337-4e76-ac4c-b3fb1a42db41 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:53.876986 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777944f0-9337-4e76-ac4c-b3fb1a42db41 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:53.894196 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:53.912629 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 262/507] 10.222.0.40 () {56 vars in 1133 bytes} [Fri Aug 5 10:13:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:53.913453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777944f0-9337-4e76-ac4c-b3fb1a42db41 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:53.946248 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b0103-0239-483b-b050-15b34c46137c None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:53.979490 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e347c-3458-4050-a0d9-f52d901d082f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:53.980104 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e347c-3458-4050-a0d9-f52d901d082f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:53.980384 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e347c-3458-4050-a0d9-f52d901d082f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:53.980661 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e347c-3458-4050-a0d9-f52d901d082f None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:53.981379 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27e347c-3458-4050-a0d9-f52d901d082f None admin] RBAC: Adding query filter params (name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:53.981686 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27e347c-3458-4050-a0d9-f52d901d082f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:53.983153 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27e347c-3458-4050-a0d9-f52d901d082f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:53.989931 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:53.991273 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 246/508] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 5 10:13:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:53.992467 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391ae9d-706b-4f61-88bb-0bbc00c60719 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:53.992746 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391ae9d-706b-4f61-88bb-0bbc00c60719 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:53.993002 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391ae9d-706b-4f61-88bb-0bbc00c60719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:54.010400 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-f391ae9d-706b-4f61-88bb-0bbc00c60719 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:54.011619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f391ae9d-706b-4f61-88bb-0bbc00c60719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:54.012175 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777944f0-9337-4e76-ac4c-b3fb1a42db41 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:54.050862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c3572-b36d-4f08-8927-ce855cfea8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:54.051535 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c3572-b36d-4f08-8927-ce855cfea8fc None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:54.051821 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c3572-b36d-4f08-8927-ce855cfea8fc None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:54.052147 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c3572-b36d-4f08-8927-ce855cfea8fc None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:54.052883 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c3572-b36d-4f08-8927-ce855cfea8fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:54.053192 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c3572-b36d-4f08-8927-ce855cfea8fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:54.054705 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c3572-b36d-4f08-8927-ce855cfea8fc None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:54.066798 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 263/509] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:13:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:54.067980 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c3572-b36d-4f08-8927-ce855cfea8fc None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:54.092864 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 247/510] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:54.094190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f391ae9d-706b-4f61-88bb-0bbc00c60719 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:54.139918 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:54.173465 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c3572-b36d-4f08-8927-ce855cfea8fc None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:54.195287 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f391ae9d-706b-4f61-88bb-0bbc00c60719 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:54.208489 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c7dc9-e7cd-4ea4-819d-1a3366d4ef22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:54.209144 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c7dc9-e7cd-4ea4-819d-1a3366d4ef22 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:54.209424 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c7dc9-e7cd-4ea4-819d-1a3366d4ef22 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:54.209694 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c7dc9-e7cd-4ea4-819d-1a3366d4ef22 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:54.210380 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216c7dc9-e7cd-4ea4-819d-1a3366d4ef22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:54.210683 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216c7dc9-e7cd-4ea4-819d-1a3366d4ef22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:54.212119 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216c7dc9-e7cd-4ea4-819d-1a3366d4ef22 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:54.219479 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:54.225408 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 264/511] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:13:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:54.226265 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c7dc9-e7cd-4ea4-819d-1a3366d4ef22 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:54.232397 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ea0c9-0447-408a-8e8c-b172414ee82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:54.233020 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ea0c9-0447-408a-8e8c-b172414ee82d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:54.233294 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ea0c9-0447-408a-8e8c-b172414ee82d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:54.233626 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ea0c9-0447-408a-8e8c-b172414ee82d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:54.301647 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256ea0c9-0447-408a-8e8c-b172414ee82d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:54.303136 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256ea0c9-0447-408a-8e8c-b172414ee82d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:54.327597 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c7dc9-e7cd-4ea4-819d-1a3366d4ef22 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:54.342492 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:54.361621 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb611020-4e34-4564-b54f-146dd26d8933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:54.362265 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb611020-4e34-4564-b54f-146dd26d8933 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:54.362567 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb611020-4e34-4564-b54f-146dd26d8933 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:54.362920 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb611020-4e34-4564-b54f-146dd26d8933 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:54.372911 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb611020-4e34-4564-b54f-146dd26d8933 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:54.374362 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb611020-4e34-4564-b54f-146dd26d8933 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:54.389796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-bb611020-4e34-4564-b54f-146dd26d8933 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:13:54.389796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:54.389796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:54.389796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:54.389796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:54.389796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:54.389796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:54.389796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:54.389796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:54.389796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:54.389796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:54.389796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:54.389796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:13:54.389796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:13:54.389796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:13:54.390653 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:13:54.390653 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:54.390653 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:54.390653 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:54.390653 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:54.390653 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:54.390653 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:54.390653 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:54.390653 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:54.390653 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:54.390653 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:54.390653 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:54.390653 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:54.390653 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:54.390653 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:54.390653 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:54.390653 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:54.391555 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:13:54.391555 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:13:54.391555 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:13:54.391555 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:54.391555 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 265/512] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:13:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:54.391831 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb611020-4e34-4564-b54f-146dd26d8933 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:54.399138 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 248/513] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:13:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 10:13:54.399964 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ea0c9-0447-408a-8e8c-b172414ee82d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:54.444290 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:54.494850 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb611020-4e34-4564-b54f-146dd26d8933 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:54.500076 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ea0c9-0447-408a-8e8c-b172414ee82d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:54.527079 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c1b401-ce91-4135-a3b7-4d013120a609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:54.527712 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1b401-ce91-4135-a3b7-4d013120a609 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:54.527987 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1b401-ce91-4135-a3b7-4d013120a609 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:54.528231 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1b401-ce91-4135-a3b7-4d013120a609 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:54.534751 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da355c0-907c-447e-8d80-21b10dc0df1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:54.535425 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da355c0-907c-447e-8d80-21b10dc0df1c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:54.535706 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da355c0-907c-447e-8d80-21b10dc0df1c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:54.535975 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da355c0-907c-447e-8d80-21b10dc0df1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:54.536212 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c1b401-ce91-4135-a3b7-4d013120a609 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:54.537633 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c1b401-ce91-4135-a3b7-4d013120a609 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:54.547139 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da355c0-907c-447e-8d80-21b10dc0df1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:54.548617 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da355c0-907c-447e-8d80-21b10dc0df1c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:54.549087 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-a0c1b401-ce91-4135-a3b7-4d013120a609 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 10:13:54.549087 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:54.549087 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:54.549087 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:54.549087 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:54.549087 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:54.549087 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:54.549087 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:54.549087 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:54.549087 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:54.549087 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:54.549087 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:54.549087 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 10:13:54.549087 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 10:13:54.549087 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 10:13:54.549087 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 10:13:54.549945 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:54.549945 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:54.549945 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 10:13:54.549945 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:13:54.549945 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 10:13:54.549945 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:13:54.549945 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:54.549945 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:54.549945 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:54.549945 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:54.549945 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:54.549945 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:54.549945 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:54.549945 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:54.549945 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:54.549945 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:54.549945 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:54.549945 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:54.550826 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:54.550826 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:54.550826 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 10:13:54.550826 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 10:13:54.550826 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 10:13:54.550826 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 10:13:54.550826 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 10:13:54.550826 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 10:13:54.550826 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 10:13:54.550826 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:54.550826 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 266/514] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:13:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:54.551411 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:54.551476 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c1b401-ce91-4135-a3b7-4d013120a609 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:54.568940 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-4da355c0-907c-447e-8d80-21b10dc0df1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:13:54.568940 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:54.568940 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:54.568940 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:54.568940 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:54.568940 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:54.568940 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:54.568940 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:54.568940 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:54.568940 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:54.568940 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:54.568940 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:54.568940 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:13:54.568940 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:13:54.568940 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:13:54.569843 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:13:54.569843 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:54.569843 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:54.569843 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:54.569843 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:54.569843 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:54.569843 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:54.569843 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:54.569843 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:54.569843 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:54.569843 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:54.569843 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:54.569843 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:54.569843 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:54.569843 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:54.569843 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:54.569843 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:54.570663 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:13:54.570663 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:13:54.570663 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:13:54.570663 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:13:54.570663 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 249/515] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:13:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:54.570945 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da355c0-907c-447e-8d80-21b10dc0df1c None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:54.655377 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c1b401-ce91-4135-a3b7-4d013120a609 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:54.665519 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:54.670534 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da355c0-907c-447e-8d80-21b10dc0df1c None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:54.690124 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac71e9f-25e5-47a0-8842-0a9101c55da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:54.690771 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac71e9f-25e5-47a0-8842-0a9101c55da5 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:54.691129 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac71e9f-25e5-47a0-8842-0a9101c55da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:54.691412 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac71e9f-25e5-47a0-8842-0a9101c55da5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:54.698108 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac71e9f-25e5-47a0-8842-0a9101c55da5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:54.699905 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac71e9f-25e5-47a0-8842-0a9101c55da5 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:54.702990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e831f7-2441-4e24-ade1-8416dd7543c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:54.703604 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e831f7-2441-4e24-ade1-8416dd7543c8 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:54.703886 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e831f7-2441-4e24-ade1-8416dd7543c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:54.704161 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e831f7-2441-4e24-ade1-8416dd7543c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:54.704849 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e831f7-2441-4e24-ade1-8416dd7543c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:54.705151 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e831f7-2441-4e24-ade1-8416dd7543c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:54.706507 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e831f7-2441-4e24-ade1-8416dd7543c8 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:54.712038 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-4ac71e9f-25e5-47a0-8842-0a9101c55da5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:13:54.712038 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:54.712038 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:54.712038 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:54.712038 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:54.712038 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:54.712038 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:54.712038 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:54.712038 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:54.712038 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:54.712038 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:54.712038 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:54.712038 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 10:13:54.712038 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 10:13:54.712038 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 10:13:54.712992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 10:13:54.712992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:54.712992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:54.712992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:54.712992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:54.712992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:54.712992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:54.712992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:54.712992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:54.712992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:54.712992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:54.712992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:54.712992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:54.712992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:54.712992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:54.712992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:54.712992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:54.713837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 10:13:54.713837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 10:13:54.713837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 10:13:54.713837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 10:13:54.713837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 10:13:54.713837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 10:13:54.713837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:13:54.713837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:54.713837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 267/516] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:13:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:54.714475 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac71e9f-25e5-47a0-8842-0a9101c55da5 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:54.724107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 250/517] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:13:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:54.724107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e831f7-2441-4e24-ade1-8416dd7543c8 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:54.774313 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:54.816996 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac71e9f-25e5-47a0-8842-0a9101c55da5 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:54.821017 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e831f7-2441-4e24-ade1-8416dd7543c8 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:54.855048 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf8539-d3bd-4488-99b8-6862637c350f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:54.855764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf8539-d3bd-4488-99b8-6862637c350f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:54.856087 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf8539-d3bd-4488-99b8-6862637c350f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:54.856361 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf8539-d3bd-4488-99b8-6862637c350f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:54.858536 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e5f254-2c79-4b62-aa91-718db1fff954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:54.859188 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e5f254-2c79-4b62-aa91-718db1fff954 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:54.859467 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e5f254-2c79-4b62-aa91-718db1fff954 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:54.859737 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e5f254-2c79-4b62-aa91-718db1fff954 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:54.860532 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e5f254-2c79-4b62-aa91-718db1fff954 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:54.860879 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e5f254-2c79-4b62-aa91-718db1fff954 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:54.862312 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e5f254-2c79-4b62-aa91-718db1fff954 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:54.868043 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cf8539-d3bd-4488-99b8-6862637c350f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:54.869513 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cf8539-d3bd-4488-99b8-6862637c350f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:54.880533 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 251/518] 10.222.0.40 () {56 vars in 1137 bytes} [Fri Aug 5 10:13:54 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:54.881571 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e5f254-2c79-4b62-aa91-718db1fff954 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:54.884926 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-a6cf8539-d3bd-4488-99b8-6862637c350f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:13:54.884926 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:54.884926 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:54.884926 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:54.884926 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:54.884926 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:54.884926 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:54.884926 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:54.884926 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:54.884926 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:54.884926 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:54.884926 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:54.884926 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:13:54.884926 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:13:54.884926 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:13:54.885775 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:13:54.885775 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:54.885775 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:54.885775 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:54.885775 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:54.885775 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:54.885775 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:54.885775 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:54.885775 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:54.885775 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:54.885775 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:54.885775 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:54.885775 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:54.885775 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:54.885775 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:54.885775 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:54.885775 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:54.886749 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:13:54.886749 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:13:54.886749 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:13:54.886749 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:54.890975 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 268/519] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:13:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:54.891652 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf8539-d3bd-4488-99b8-6862637c350f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:54.916502 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:54.978157 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e5f254-2c79-4b62-aa91-718db1fff954 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:54.991382 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:54.992352 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf8539-d3bd-4488-99b8-6862637c350f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:55.017083 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29215cfe-bf1c-44c1-b973-5e839fd08a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:55.017704 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29215cfe-bf1c-44c1-b973-5e839fd08a5c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:55.017985 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29215cfe-bf1c-44c1-b973-5e839fd08a5c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:55.018243 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29215cfe-bf1c-44c1-b973-5e839fd08a5c None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:55.018973 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29215cfe-bf1c-44c1-b973-5e839fd08a5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:55.019279 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29215cfe-bf1c-44c1-b973-5e839fd08a5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:55.020670 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29215cfe-bf1c-44c1-b973-5e839fd08a5c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:55.025860 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e611a62c-1576-49b7-a7f8-507151ae68b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:55.026412 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e611a62c-1576-49b7-a7f8-507151ae68b3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:55.026688 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e611a62c-1576-49b7-a7f8-507151ae68b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:55.026990 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e611a62c-1576-49b7-a7f8-507151ae68b3 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:55.027725 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e611a62c-1576-49b7-a7f8-507151ae68b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:55.028024 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e611a62c-1576-49b7-a7f8-507151ae68b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:55.029553 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e611a62c-1576-49b7-a7f8-507151ae68b3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:55.031990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 252/520] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:13:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:55.032781 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29215cfe-bf1c-44c1-b973-5e839fd08a5c None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:55.051822 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 269/521] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 10:13:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:55.052681 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e611a62c-1576-49b7-a7f8-507151ae68b3 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:55.134243 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29215cfe-bf1c-44c1-b973-5e839fd08a5c None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:55.135339 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:55.154524 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e611a62c-1576-49b7-a7f8-507151ae68b3 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:55.170404 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c79d0-936d-4e16-b2c6-6452b94e2e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:55.171087 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c79d0-936d-4e16-b2c6-6452b94e2e26 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:55.171350 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c79d0-936d-4e16-b2c6-6452b94e2e26 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:55.171618 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c79d0-936d-4e16-b2c6-6452b94e2e26 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:55.179213 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030c79d0-936d-4e16-b2c6-6452b94e2e26 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:55.180698 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030c79d0-936d-4e16-b2c6-6452b94e2e26 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:55.192023 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab0ceb-d7e1-4759-96e9-e906753049b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:55.192639 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab0ceb-d7e1-4759-96e9-e906753049b6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:55.192912 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab0ceb-d7e1-4759-96e9-e906753049b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:55.193176 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab0ceb-d7e1-4759-96e9-e906753049b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:55.193783 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-030c79d0-936d-4e16-b2c6-6452b94e2e26 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 05 10:13:55.193783 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:55.193783 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:55.193783 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:55.193783 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:55.193783 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:55.193783 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:55.193783 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:55.193783 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:55.193783 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:55.193783 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:55.193783 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:55.193783 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 10:13:55.193783 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 10:13:55.193783 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 10:13:55.193783 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 10:13:55.194675 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:55.194675 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:55.194675 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 10:13:55.194675 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:13:55.194675 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 10:13:55.194675 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:13:55.194675 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:55.194675 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:55.194675 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:55.194675 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:55.194675 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:55.194675 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:55.194675 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:55.194675 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:55.194675 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:55.194675 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:55.194675 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:55.194675 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:55.195799 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:55.195799 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:55.195799 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 10:13:55.195799 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 10:13:55.195799 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 10:13:55.195799 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 10:13:55.195799 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 10:13:55.195799 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 10:13:55.195799 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 05 10:13:55.195799 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:13:55.195799 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 253/522] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 10:13:55 2022] GET /identity/v3/users/neutron => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:55.195799 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c79d0-936d-4e16-b2c6-6452b94e2e26 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:55.227450 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:55.264590 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ab0ceb-d7e1-4759-96e9-e906753049b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:55.266090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ab0ceb-d7e1-4759-96e9-e906753049b6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:55.295407 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c79d0-936d-4e16-b2c6-6452b94e2e26 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:55.328375 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8972b-63c8-4be4-aaaf-470dcd5e8a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:55.329019 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8972b-63c8-4be4-aaaf-470dcd5e8a50 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:55.329287 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8972b-63c8-4be4-aaaf-470dcd5e8a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:55.329561 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8972b-63c8-4be4-aaaf-470dcd5e8a50 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:55.330251 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc8972b-63c8-4be4-aaaf-470dcd5e8a50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:55.330555 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc8972b-63c8-4be4-aaaf-470dcd5e8a50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:55.331988 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc8972b-63c8-4be4-aaaf-470dcd5e8a50 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:55.343908 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 254/523] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:13:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:55.344682 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8972b-63c8-4be4-aaaf-470dcd5e8a50 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:55.345863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:55.366273 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 270/524] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:13:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 10:13:55.367190 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab0ceb-d7e1-4759-96e9-e906753049b6 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:55.435172 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:55.445396 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8972b-63c8-4be4-aaaf-470dcd5e8a50 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:55.471405 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab0ceb-d7e1-4759-96e9-e906753049b6 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:55.479700 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca158441-e5ba-427b-b8f1-39ce2794192e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:55.480325 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca158441-e5ba-427b-b8f1-39ce2794192e None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:55.480607 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca158441-e5ba-427b-b8f1-39ce2794192e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:55.480877 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca158441-e5ba-427b-b8f1-39ce2794192e None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:55.481544 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca158441-e5ba-427b-b8f1-39ce2794192e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:55.483223 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca158441-e5ba-427b-b8f1-39ce2794192e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:55.504368 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 255/525] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 5 10:13:55 2022] POST /identity/v3/users => generated 247 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:13:55.504908 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac92bc3-ef2a-4982-8035-fceb02a8b800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:55.505160 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca158441-e5ba-427b-b8f1-39ce2794192e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:55.505542 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac92bc3-ef2a-4982-8035-fceb02a8b800 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:55.505854 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac92bc3-ef2a-4982-8035-fceb02a8b800 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:55.506083 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac92bc3-ef2a-4982-8035-fceb02a8b800 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:55.507148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac92bc3-ef2a-4982-8035-fceb02a8b800 None admin] RBAC: Adding query filter params (role.id=3a2b4319820a488198dabee34420cdc7, scope.project.id=36b8b505012d4d0bac54845b6799e743, user.id=ac0a8cf7f8b74e88a04444ec38547528) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:55.507749 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac92bc3-ef2a-4982-8035-fceb02a8b800 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:55.510196 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac92bc3-ef2a-4982-8035-fceb02a8b800 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:55.518083 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 271/526] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:13:55 2022] GET /identity/v3/role_assignments?user.id=ac0a8cf7f8b74e88a04444ec38547528&scope.project.id=36b8b505012d4d0bac54845b6799e743&role.id=3a2b4319820a488198dabee34420cdc7 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:55.518802 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac92bc3-ef2a-4982-8035-fceb02a8b800 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:55.589093 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:55.646251 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca158441-e5ba-427b-b8f1-39ce2794192e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:55.654015 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac92bc3-ef2a-4982-8035-fceb02a8b800 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:55.692213 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf46c986-bf9e-4fb5-986f-8ee54588ad50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:55.692213 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf46c986-bf9e-4fb5-986f-8ee54588ad50 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:55.692213 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf46c986-bf9e-4fb5-986f-8ee54588ad50 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:55.692213 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf46c986-bf9e-4fb5-986f-8ee54588ad50 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:55.693144 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf46c986-bf9e-4fb5-986f-8ee54588ad50 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:55.693308 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf46c986-bf9e-4fb5-986f-8ee54588ad50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:55.694783 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9c577e-3dcf-490d-8af1-9c522c3199ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:55.695191 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf46c986-bf9e-4fb5-986f-8ee54588ad50 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:55.695264 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c577e-3dcf-490d-8af1-9c522c3199ea None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:55.695264 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c577e-3dcf-490d-8af1-9c522c3199ea None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:55.695526 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c577e-3dcf-490d-8af1-9c522c3199ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:55.705769 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9c577e-3dcf-490d-8af1-9c522c3199ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:55.707264 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9c577e-3dcf-490d-8af1-9c522c3199ea None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:55.715483 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 256/527] 10.222.0.40 () {56 vars in 1139 bytes} [Fri Aug 5 10:13:55 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:55.717308 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf46c986-bf9e-4fb5-986f-8ee54588ad50 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:55.720545 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-5a9c577e-3dcf-490d-8af1-9c522c3199ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:13:55.720545 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:55.720545 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:55.720545 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:55.720545 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:55.720545 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:55.720545 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:55.720545 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:55.720545 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:55.720545 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:55.720545 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:55.720545 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:55.720545 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 10:13:55.720545 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 10:13:55.720545 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 10:13:55.721520 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 10:13:55.721520 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:55.721520 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:55.721520 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:55.721520 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:55.721520 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:55.721520 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:55.721520 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:55.721520 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:55.721520 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:55.721520 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:55.721520 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:55.721520 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:55.721520 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:55.721520 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:55.721520 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:55.721520 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:55.722543 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 10:13:55.722543 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 10:13:55.722543 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 10:13:55.722543 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 10:13:55.722543 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 10:13:55.722543 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 10:13:55.722543 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:13:55.722543 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:55.722543 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 272/528] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:13:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:55.724986 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9c577e-3dcf-490d-8af1-9c522c3199ea None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:55.732819 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:55.853031 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf46c986-bf9e-4fb5-986f-8ee54588ad50 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:55.853531 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9c577e-3dcf-490d-8af1-9c522c3199ea None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:55.863485 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:55.898761 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f518c450-90b4-4a4b-8555-f9c33f560e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:55.899213 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79fb367-4e1a-4d3b-a4aa-e0e9a635f349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:55.899840 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518c450-90b4-4a4b-8555-f9c33f560e40 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:55.900054 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79fb367-4e1a-4d3b-a4aa-e0e9a635f349 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:55.900329 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79fb367-4e1a-4d3b-a4aa-e0e9a635f349 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:55.900609 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518c450-90b4-4a4b-8555-f9c33f560e40 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:55.900714 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79fb367-4e1a-4d3b-a4aa-e0e9a635f349 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:55.901095 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518c450-90b4-4a4b-8555-f9c33f560e40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:55.913854 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79fb367-4e1a-4d3b-a4aa-e0e9a635f349 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:55.916856 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79fb367-4e1a-4d3b-a4aa-e0e9a635f349 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:55.934582 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-b79fb367-4e1a-4d3b-a4aa-e0e9a635f349 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:13:55.934582 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:55.934582 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:55.934582 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:55.934582 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:55.934582 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:55.934582 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:55.934582 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:55.934582 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:55.934582 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:55.934582 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:55.934582 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:55.934582 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:13:55.934582 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:13:55.934582 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:13:55.935853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:13:55.935853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:55.935853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:55.935853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:55.935853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:55.935853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:55.935853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:55.935853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:55.935853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:55.935853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:55.935853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:55.935853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:55.935853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:55.935853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:55.935853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:55.935853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:55.935853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:55.936888 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:13:55.936888 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:13:55.936888 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:13:55.936888 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:13:55.938783 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 257/529] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:13:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:55.939847 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79fb367-4e1a-4d3b-a4aa-e0e9a635f349 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:55.980435 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f518c450-90b4-4a4b-8555-f9c33f560e40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:55.982176 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f518c450-90b4-4a4b-8555-f9c33f560e40 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:56.002364 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:56.045419 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79fb367-4e1a-4d3b-a4aa-e0e9a635f349 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:56.085046 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4b4ff-7194-4f1a-b4fc-b2d526703da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:56.086075 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4b4ff-7194-4f1a-b4fc-b2d526703da5 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:56.086843 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4b4ff-7194-4f1a-b4fc-b2d526703da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:56.087266 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4b4ff-7194-4f1a-b4fc-b2d526703da5 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:56.088392 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa4b4ff-7194-4f1a-b4fc-b2d526703da5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:56.088796 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa4b4ff-7194-4f1a-b4fc-b2d526703da5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:56.091390 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa4b4ff-7194-4f1a-b4fc-b2d526703da5 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:56.098650 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 273/530] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:13:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 10:13:56.099800 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f518c450-90b4-4a4b-8555-f9c33f560e40 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:56.109497 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:56.114057 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 258/531] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 10:13:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:56.115598 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4b4ff-7194-4f1a-b4fc-b2d526703da5 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:56.230489 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4b4ff-7194-4f1a-b4fc-b2d526703da5 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:56.234517 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f518c450-90b4-4a4b-8555-f9c33f560e40 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:56.250349 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:56.269172 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b273a3-5f52-4004-871b-5aee3b82b54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:56.270158 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b273a3-5f52-4004-871b-5aee3b82b54d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:56.270671 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b273a3-5f52-4004-871b-5aee3b82b54d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:56.271186 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b273a3-5f52-4004-871b-5aee3b82b54d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:56.278948 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c98b0ee-7ad6-4b31-b4ed-590df99f16ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:56.279584 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c98b0ee-7ad6-4b31-b4ed-590df99f16ef None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:56.279862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c98b0ee-7ad6-4b31-b4ed-590df99f16ef None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:56.280132 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c98b0ee-7ad6-4b31-b4ed-590df99f16ef None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:56.280825 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c98b0ee-7ad6-4b31-b4ed-590df99f16ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:56.281127 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c98b0ee-7ad6-4b31-b4ed-590df99f16ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:56.283311 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c98b0ee-7ad6-4b31-b4ed-590df99f16ef None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:56.283924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b273a3-5f52-4004-871b-5aee3b82b54d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:56.285564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b273a3-5f52-4004-871b-5aee3b82b54d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:56.298396 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 274/532] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:13:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:56.299411 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c98b0ee-7ad6-4b31-b4ed-590df99f16ef None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:56.303136 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-56b273a3-5f52-4004-871b-5aee3b82b54d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:13:56.303136 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:56.303136 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:56.303136 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:56.303136 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:56.303136 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:56.303136 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:56.303136 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:56.303136 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:56.303136 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:56.303136 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:56.303136 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:56.303136 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:13:56.303136 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:13:56.303136 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:13:56.303967 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:13:56.303967 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:56.303967 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:56.303967 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:56.303967 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:56.303967 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:56.303967 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:56.303967 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:56.303967 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:56.303967 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:56.303967 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:56.303967 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:56.303967 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:56.303967 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:56.303967 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:56.303967 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:56.303967 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:56.304823 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:13:56.304823 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:13:56.304823 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:13:56.304823 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:13:56.306762 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 259/533] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:13:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:56.307962 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b273a3-5f52-4004-871b-5aee3b82b54d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:56.371391 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:56.421192 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c98b0ee-7ad6-4b31-b4ed-590df99f16ef None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:56.430557 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b273a3-5f52-4004-871b-5aee3b82b54d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:56.462517 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008a9bda-e012-4940-a740-ac8065f3d95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:56.463319 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008a9bda-e012-4940-a740-ac8065f3d95d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:56.463693 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008a9bda-e012-4940-a740-ac8065f3d95d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:56.464146 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008a9bda-e012-4940-a740-ac8065f3d95d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:56.464992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008a9bda-e012-4940-a740-ac8065f3d95d None admin] RBAC: Adding query filter params (role.id=3a2b4319820a488198dabee34420cdc7, scope.project.id=36b8b505012d4d0bac54845b6799e743, user.id=b938f1f41db94b0eafe95bc8eeaae175) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:56.465454 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008a9bda-e012-4940-a740-ac8065f3d95d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:56.467040 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008a9bda-e012-4940-a740-ac8065f3d95d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:56.468433 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f446f4ec-c7b2-4cba-a4ff-a6c4dc609b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:56.469471 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f446f4ec-c7b2-4cba-a4ff-a6c4dc609b24 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:56.469909 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f446f4ec-c7b2-4cba-a4ff-a6c4dc609b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:56.470775 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f446f4ec-c7b2-4cba-a4ff-a6c4dc609b24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:56.473403 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 275/534] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:13:56 2022] GET /identity/v3/role_assignments?user.id=b938f1f41db94b0eafe95bc8eeaae175&scope.project.id=36b8b505012d4d0bac54845b6799e743&role.id=3a2b4319820a488198dabee34420cdc7 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:56.474341 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008a9bda-e012-4940-a740-ac8065f3d95d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:56.480207 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f446f4ec-c7b2-4cba-a4ff-a6c4dc609b24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:56.482155 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f446f4ec-c7b2-4cba-a4ff-a6c4dc609b24 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:56.499854 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-f446f4ec-c7b2-4cba-a4ff-a6c4dc609b24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:13:56.499854 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:56.499854 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:56.499854 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:56.499854 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:56.499854 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:56.499854 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:56.499854 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:56.499854 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:56.499854 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:56.499854 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:56.499854 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:56.499854 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 10:13:56.499854 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 10:13:56.499854 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 10:13:56.500770 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 10:13:56.500770 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:56.500770 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:56.500770 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:56.500770 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:56.500770 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:56.500770 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:56.500770 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:56.500770 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:56.500770 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:56.500770 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:56.500770 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:56.500770 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:56.500770 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:56.500770 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:56.500770 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:56.500770 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:56.501622 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 10:13:56.501622 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 10:13:56.501622 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 10:13:56.501622 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 10:13:56.501622 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 10:13:56.501622 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 10:13:56.501622 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:13:56.501622 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:13:56.503886 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 260/535] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:13:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:56.510964 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f446f4ec-c7b2-4cba-a4ff-a6c4dc609b24 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:56.549919 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:56.669290 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008a9bda-e012-4940-a740-ac8065f3d95d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:56.680971 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f446f4ec-c7b2-4cba-a4ff-a6c4dc609b24 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:56.717602 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:56.725267 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c6dc09-5bf3-49c2-b17a-3e556a25789a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:56.725900 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6dc09-5bf3-49c2-b17a-3e556a25789a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:56.726181 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6dc09-5bf3-49c2-b17a-3e556a25789a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:56.726449 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6dc09-5bf3-49c2-b17a-3e556a25789a None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:56.727238 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c6dc09-5bf3-49c2-b17a-3e556a25789a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:56.727538 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c6dc09-5bf3-49c2-b17a-3e556a25789a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:56.729081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c6dc09-5bf3-49c2-b17a-3e556a25789a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:56.740198 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e32480-67cf-49ee-a97b-83d5ebbbe977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:56.740842 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e32480-67cf-49ee-a97b-83d5ebbbe977 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:56.741119 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e32480-67cf-49ee-a97b-83d5ebbbe977 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:56.741389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e32480-67cf-49ee-a97b-83d5ebbbe977 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:56.742145 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e32480-67cf-49ee-a97b-83d5ebbbe977 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:56.742449 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e32480-67cf-49ee-a97b-83d5ebbbe977 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:56.743101 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 261/536] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 10:13:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:56.743899 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e32480-67cf-49ee-a97b-83d5ebbbe977 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:56.752444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c6dc09-5bf3-49c2-b17a-3e556a25789a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:56.761474 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 276/537] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:13:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:56.766962 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e32480-67cf-49ee-a97b-83d5ebbbe977 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:56.866411 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:56.869286 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c6dc09-5bf3-49c2-b17a-3e556a25789a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:56.909215 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5874a2-fb7d-48e1-81cc-fb357b07b144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:56.909854 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5874a2-fb7d-48e1-81cc-fb357b07b144 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:56.910141 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5874a2-fb7d-48e1-81cc-fb357b07b144 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:56.910414 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5874a2-fb7d-48e1-81cc-fb357b07b144 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:56.911241 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5874a2-fb7d-48e1-81cc-fb357b07b144 None admin] RBAC: Adding query filter params (role.id=3a2b4319820a488198dabee34420cdc7, scope.project.id=36b8b505012d4d0bac54845b6799e743, user.id=f70369fd5b334d888de0c091d8885eb3) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:56.911549 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5874a2-fb7d-48e1-81cc-fb357b07b144 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:56.913345 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5874a2-fb7d-48e1-81cc-fb357b07b144 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:56.919688 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 262/538] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:13:56 2022] GET /identity/v3/role_assignments?user.id=f70369fd5b334d888de0c091d8885eb3&scope.project.id=36b8b505012d4d0bac54845b6799e743&role.id=3a2b4319820a488198dabee34420cdc7 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:57.058120 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e32480-67cf-49ee-a97b-83d5ebbbe977 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:57.114616 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c5d12-9b40-46a3-ba83-21d2e9480537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:57.115311 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c5d12-9b40-46a3-ba83-21d2e9480537 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:57.115588 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c5d12-9b40-46a3-ba83-21d2e9480537 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:57.115855 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c5d12-9b40-46a3-ba83-21d2e9480537 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:57.123482 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4c5d12-9b40-46a3-ba83-21d2e9480537 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:57.124932 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4c5d12-9b40-46a3-ba83-21d2e9480537 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:57.142672 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-df4c5d12-9b40-46a3-ba83-21d2e9480537 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:13:57.142672 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:57.142672 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:57.142672 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:57.142672 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:57.142672 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:57.142672 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:57.142672 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:57.142672 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:57.142672 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:57.142672 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:57.142672 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:57.142672 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 10:13:57.142672 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 10:13:57.142672 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 10:13:57.143616 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 10:13:57.143616 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:57.143616 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:57.143616 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:57.143616 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:57.143616 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:57.143616 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:57.143616 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:57.143616 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:57.143616 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:57.143616 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:57.143616 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:57.143616 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:57.143616 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:57.143616 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:57.143616 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:57.143616 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:57.144451 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 10:13:57.144451 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 10:13:57.144451 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 10:13:57.144451 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 10:13:57.144451 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 10:13:57.144451 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 10:13:57.144451 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:13:57.144451 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:57.144451 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 277/539] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:13:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:57.151343 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5874a2-fb7d-48e1-81cc-fb357b07b144 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:57.288073 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5874a2-fb7d-48e1-81cc-fb357b07b144 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:57.336589 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac88337-9169-4a0d-ba48-5d2fc687b50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:57.337234 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac88337-9169-4a0d-ba48-5d2fc687b50f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:57.337512 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac88337-9169-4a0d-ba48-5d2fc687b50f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:57.337778 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac88337-9169-4a0d-ba48-5d2fc687b50f None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:57.338540 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac88337-9169-4a0d-ba48-5d2fc687b50f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:57.338839 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac88337-9169-4a0d-ba48-5d2fc687b50f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:57.340549 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac88337-9169-4a0d-ba48-5d2fc687b50f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:57.357722 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 263/540] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 10:13:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:57.363220 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c5d12-9b40-46a3-ba83-21d2e9480537 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:57.419192 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:57.487809 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c5d12-9b40-46a3-ba83-21d2e9480537 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:57.533916 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4a9648-d5df-44cb-acdd-c391abb70700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:57.534541 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a9648-d5df-44cb-acdd-c391abb70700 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:57.534823 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a9648-d5df-44cb-acdd-c391abb70700 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:57.535110 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a9648-d5df-44cb-acdd-c391abb70700 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:57.535906 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4a9648-d5df-44cb-acdd-c391abb70700 None admin] RBAC: Adding query filter params (role.id=3a2b4319820a488198dabee34420cdc7, scope.project.id=36b8b505012d4d0bac54845b6799e743, user.id=a7deb94069a449269cb03406232174d3) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:57.536213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4a9648-d5df-44cb-acdd-c391abb70700 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:57.537649 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4a9648-d5df-44cb-acdd-c391abb70700 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:57.544144 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 278/541] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:13:57 2022] GET /identity/v3/role_assignments?user.id=a7deb94069a449269cb03406232174d3&scope.project.id=36b8b505012d4d0bac54845b6799e743&role.id=3a2b4319820a488198dabee34420cdc7 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:57.752725 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf179fe-5b79-40b3-a812-bd9d6eacb17f None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:57.753441 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf179fe-5b79-40b3-a812-bd9d6eacb17f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:57.753441 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf179fe-5b79-40b3-a812-bd9d6eacb17f None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:57.755025 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 264/542] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:57.760298 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00612c58-1665-48d5-952f-f6b9fccb2e2b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:57.760867 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00612c58-1665-48d5-952f-f6b9fccb2e2b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:57.760867 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00612c58-1665-48d5-952f-f6b9fccb2e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:57.777452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef9e6c9-0a7b-4807-9c0a-1200b53dce3a None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:57.777452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef9e6c9-0a7b-4807-9c0a-1200b53dce3a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:57.777452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef9e6c9-0a7b-4807-9c0a-1200b53dce3a None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:57.778933 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 265/543] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:57.783888 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-00612c58-1665-48d5-952f-f6b9fccb2e2b None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:57.785426 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-00612c58-1665-48d5-952f-f6b9fccb2e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:57.800680 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ba05f-c9c0-4646-9e75-81c17c1fbb3a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:57.800855 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ba05f-c9c0-4646-9e75-81c17c1fbb3a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:57.801137 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ba05f-c9c0-4646-9e75-81c17c1fbb3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:57.847952 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-995ba05f-c9c0-4646-9e75-81c17c1fbb3a None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:57.849161 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-995ba05f-c9c0-4646-9e75-81c17c1fbb3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:57.866429 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:57.874866 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 279/544] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:57.906954 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00612c58-1665-48d5-952f-f6b9fccb2e2b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:57.957523 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 266/545] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:57.964162 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:57.996296 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ba05f-c9c0-4646-9e75-81c17c1fbb3a None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:58.048659 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00612c58-1665-48d5-952f-f6b9fccb2e2b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:58.091933 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a939cf-bc62-4d1a-b9d6-c200946ffb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:58.092554 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a939cf-bc62-4d1a-b9d6-c200946ffb13 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:58.092838 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a939cf-bc62-4d1a-b9d6-c200946ffb13 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:58.093108 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a939cf-bc62-4d1a-b9d6-c200946ffb13 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:58.103402 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a939cf-bc62-4d1a-b9d6-c200946ffb13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:58.108373 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-97a939cf-bc62-4d1a-b9d6-c200946ffb13 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 10:13:58.108373 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:58.108373 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:58.108373 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:58.108373 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:58.108373 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:58.108373 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:58.108373 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:58.108373 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:58.108373 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:58.108373 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:58.108373 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:58.108373 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:58.108373 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:58.108373 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:58.108373 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:58.109270 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:58.109270 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:58.109270 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:58.109270 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:58.109270 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:58.109270 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:58.109270 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:58.109270 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:58.109270 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:58.109270 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:58.109270 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:58.109270 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:58.109270 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:58.109270 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:58.109270 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:58.109270 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:58.110062 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:58.110062 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:58.110062 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:58.110062 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:58.110062 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:58.110062 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:58.110062 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:58.110062 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:58.110062 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:58.110062 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:58.110062 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 10:13:58.110062 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:58.110062 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 280/546] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 10:13:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:58.115463 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a939cf-bc62-4d1a-b9d6-c200946ffb13 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:58.193101 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ba05f-c9c0-4646-9e75-81c17c1fbb3a None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:58.223772 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a939cf-bc62-4d1a-b9d6-c200946ffb13 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:58.226127 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:58.261967 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da2595b-2adc-45a9-a744-56eeeecb2d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:58.262598 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da2595b-2adc-45a9-a744-56eeeecb2d77 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:58.265178 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da2595b-2adc-45a9-a744-56eeeecb2d77 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:58.265178 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da2595b-2adc-45a9-a744-56eeeecb2d77 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:58.265178 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da2595b-2adc-45a9-a744-56eeeecb2d77 None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:58.265178 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da2595b-2adc-45a9-a744-56eeeecb2d77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:58.265681 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da2595b-2adc-45a9-a744-56eeeecb2d77 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:58.267022 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f96f486-4491-4516-b735-f5df9eeed7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:58.267913 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96f486-4491-4516-b735-f5df9eeed7f3 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:58.268384 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96f486-4491-4516-b735-f5df9eeed7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:58.268850 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96f486-4491-4516-b735-f5df9eeed7f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:58.274404 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 281/547] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 5 10:13:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:13:58.280212 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da2595b-2adc-45a9-a744-56eeeecb2d77 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:58.291796 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f96f486-4491-4516-b735-f5df9eeed7f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:58.297859 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-8f96f486-4491-4516-b735-f5df9eeed7f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 10:13:58.297859 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:58.297859 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:58.297859 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:58.297859 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:58.297859 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:58.297859 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:58.297859 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:58.297859 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:58.297859 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:58.297859 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:58.297859 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:58.297859 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:58.297859 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:58.297859 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:58.297859 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:58.298753 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:58.298753 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:58.298753 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:58.298753 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:58.298753 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:58.298753 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:58.298753 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:58.298753 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:58.298753 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:58.298753 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:58.298753 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:58.298753 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:58.298753 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:58.298753 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:58.298753 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:58.298753 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:58.299564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:58.299564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:58.299564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:58.299564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:58.299564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:58.299564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:58.299564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:58.299564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:58.299564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:58.299564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:58.299564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 10:13:58.299564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:13:58.306647 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 267/548] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 10:13:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:13:58.310998 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f96f486-4491-4516-b735-f5df9eeed7f3 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:58.318961 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:58.389495 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da2595b-2adc-45a9-a744-56eeeecb2d77 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:58.440717 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f368babd-cd26-4d4d-95c6-34e855928ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:58.441373 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f368babd-cd26-4d4d-95c6-34e855928ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:58.441649 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f368babd-cd26-4d4d-95c6-34e855928ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:58.441919 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f368babd-cd26-4d4d-95c6-34e855928ee5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:58.465362 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f368babd-cd26-4d4d-95c6-34e855928ee5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:58.467476 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f368babd-cd26-4d4d-95c6-34e855928ee5 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:58.486152 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-f368babd-cd26-4d4d-95c6-34e855928ee5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:13:58.486152 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:58.486152 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:58.486152 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:58.486152 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:58.486152 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:58.486152 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:58.486152 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:58.486152 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:58.486152 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:58.486152 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:58.486152 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:58.486152 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:13:58.486152 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:13:58.486152 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:13:58.487086 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:13:58.487086 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:58.487086 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:58.487086 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:58.487086 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:58.487086 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:58.487086 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:58.487086 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:58.487086 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:58.487086 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:58.487086 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:58.487086 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:58.487086 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:58.487086 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:58.487086 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:58.487086 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:58.487086 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:58.488582 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:13:58.488582 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:13:58.488582 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:13:58.488582 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:58.491240 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f96f486-4491-4516-b735-f5df9eeed7f3 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:58.501460 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 282/549] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:13:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:58.502670 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f368babd-cd26-4d4d-95c6-34e855928ee5 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:58.544650 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52b57d-6fd2-4551-b9e8-85db56bec6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:58.545634 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52b57d-6fd2-4551-b9e8-85db56bec6a4 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:58.546111 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52b57d-6fd2-4551-b9e8-85db56bec6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:58.546598 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52b57d-6fd2-4551-b9e8-85db56bec6a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:58.547550 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f52b57d-6fd2-4551-b9e8-85db56bec6a4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:58.548070 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f52b57d-6fd2-4551-b9e8-85db56bec6a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:58.549663 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f52b57d-6fd2-4551-b9e8-85db56bec6a4 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:58.552361 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:58.560091 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 268/550] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 5 10:13:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:13:58.566066 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52b57d-6fd2-4551-b9e8-85db56bec6a4 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:58.656861 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f368babd-cd26-4d4d-95c6-34e855928ee5 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:58.711829 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076d5d24-cc84-4956-893b-e9d454d8ac92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:58.712735 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076d5d24-cc84-4956-893b-e9d454d8ac92 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:58.713151 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076d5d24-cc84-4956-893b-e9d454d8ac92 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:58.713564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:58.714612 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076d5d24-cc84-4956-893b-e9d454d8ac92 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:58.715805 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076d5d24-cc84-4956-893b-e9d454d8ac92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:58.717420 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076d5d24-cc84-4956-893b-e9d454d8ac92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:58.720084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076d5d24-cc84-4956-893b-e9d454d8ac92 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:58.747969 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 283/551] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:58.752012 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076d5d24-cc84-4956-893b-e9d454d8ac92 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:58.779964 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52b57d-6fd2-4551-b9e8-85db56bec6a4 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:58.844529 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94ea4cc-b2d5-4b73-b330-1506745ef3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:58.845527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ea4cc-b2d5-4b73-b330-1506745ef3d4 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:58.845976 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ea4cc-b2d5-4b73-b330-1506745ef3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:58.846395 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ea4cc-b2d5-4b73-b330-1506745ef3d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:58.869424 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94ea4cc-b2d5-4b73-b330-1506745ef3d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:58.869977 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:58.872601 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94ea4cc-b2d5-4b73-b330-1506745ef3d4 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:58.894498 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076d5d24-cc84-4956-893b-e9d454d8ac92 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:58.899447 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-c94ea4cc-b2d5-4b73-b330-1506745ef3d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:13:58.899447 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:58.899447 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:58.899447 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:58.899447 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:58.899447 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:58.899447 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:58.899447 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:58.899447 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:58.899447 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:58.899447 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:58.899447 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:58.899447 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:13:58.899447 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:13:58.899447 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:13:58.900525 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:13:58.900525 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:58.900525 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:58.900525 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:58.900525 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:58.900525 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:58.900525 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:58.900525 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:58.900525 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:58.900525 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:58.900525 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:58.900525 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:58.900525 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:58.900525 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:58.900525 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:58.900525 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:58.900525 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:58.901763 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:13:58.901763 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:13:58.901763 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:13:58.901763 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:13:58.911523 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 269/552] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:13:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:13:58.916559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94ea4cc-b2d5-4b73-b330-1506745ef3d4 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:58.931479 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed0f11-2e89-4a18-a7dd-0f3b70025ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:58.932129 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed0f11-2e89-4a18-a7dd-0f3b70025ea5 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:58.932407 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed0f11-2e89-4a18-a7dd-0f3b70025ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:58.932675 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed0f11-2e89-4a18-a7dd-0f3b70025ea5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:58.941689 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ed0f11-2e89-4a18-a7dd-0f3b70025ea5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:58.943477 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ed0f11-2e89-4a18-a7dd-0f3b70025ea5 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:58.958370 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-c0ed0f11-2e89-4a18-a7dd-0f3b70025ea5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 05 10:13:58.958370 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:58.958370 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:58.958370 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:58.958370 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:58.958370 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:58.958370 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:58.958370 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:58.958370 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:58.958370 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:58.958370 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:58.958370 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:58.958370 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 10:13:58.958370 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 10:13:58.958370 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 10:13:58.958370 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 10:13:58.959517 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:58.959517 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:58.959517 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 10:13:58.959517 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:13:58.959517 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 10:13:58.959517 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:13:58.959517 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:58.959517 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:58.959517 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:58.959517 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:58.959517 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:58.959517 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:58.959517 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:58.959517 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:58.959517 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:58.959517 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:58.959517 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:58.959517 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:58.960582 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:58.960582 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:58.960582 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 10:13:58.960582 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 10:13:58.960582 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 10:13:58.960582 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 10:13:58.960582 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 10:13:58.960582 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 10:13:58.960582 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 05 10:13:58.960582 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:58.960582 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 284/553] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:13:58 2022] GET /identity/v3/users/glance => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:13:58.965377 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed0f11-2e89-4a18-a7dd-0f3b70025ea5 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:59.074928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed0f11-2e89-4a18-a7dd-0f3b70025ea5 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:59.097172 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94ea4cc-b2d5-4b73-b330-1506745ef3d4 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:59.105275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d8646-e605-4ff0-b5a4-75809352f6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:59.105902 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d8646-e605-4ff0-b5a4-75809352f6f6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:59.106184 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d8646-e605-4ff0-b5a4-75809352f6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:59.106495 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d8646-e605-4ff0-b5a4-75809352f6f6 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:59.107340 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d8646-e605-4ff0-b5a4-75809352f6f6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:59.107646 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d8646-e605-4ff0-b5a4-75809352f6f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:59.109167 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d8646-e605-4ff0-b5a4-75809352f6f6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:59.116727 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:59.117911 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:59.130465 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 285/554] 10.222.0.40 () {56 vars in 1137 bytes} [Fri Aug 5 10:13:58 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:13:59.136607 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a812d46-b0cb-4626-8201-7d57c6b4e5a1 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:59.136901 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a812d46-b0cb-4626-8201-7d57c6b4e5a1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:59.137178 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a812d46-b0cb-4626-8201-7d57c6b4e5a1 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:59.137640 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca73dbfc-ffe1-4e19-b4da-879f2ffb5214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:59.138814 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 286/555] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:59.142843 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a812d46-b0cb-4626-8201-7d57c6b4e5a1 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:59.143411 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73dbfc-ffe1-4e19-b4da-879f2ffb5214 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:59.143932 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73dbfc-ffe1-4e19-b4da-879f2ffb5214 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:59.144370 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73dbfc-ffe1-4e19-b4da-879f2ffb5214 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:59.145236 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca73dbfc-ffe1-4e19-b4da-879f2ffb5214 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:59.145679 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca73dbfc-ffe1-4e19-b4da-879f2ffb5214 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:59.147239 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca73dbfc-ffe1-4e19-b4da-879f2ffb5214 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:59.161268 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 270/556] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:13:59.162993 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fea0fb-c7cb-4cb2-8aba-6a88bdff8f2d None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:59.163590 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fea0fb-c7cb-4cb2-8aba-6a88bdff8f2d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:59.164057 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fea0fb-c7cb-4cb2-8aba-6a88bdff8f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:59.187175 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-86fea0fb-c7cb-4cb2-8aba-6a88bdff8f2d None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:59.188862 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-86fea0fb-c7cb-4cb2-8aba-6a88bdff8f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:59.248787 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a812d46-b0cb-4626-8201-7d57c6b4e5a1 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:59.266997 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 271/557] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:59.268314 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fea0fb-c7cb-4cb2-8aba-6a88bdff8f2d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:59.292328 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fdd2b6-3efb-4201-93bd-db3389bb984d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:59.292986 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fdd2b6-3efb-4201-93bd-db3389bb984d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:59.293372 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fdd2b6-3efb-4201-93bd-db3389bb984d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:59.293641 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fdd2b6-3efb-4201-93bd-db3389bb984d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:59.306649 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fdd2b6-3efb-4201-93bd-db3389bb984d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:59.308451 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fdd2b6-3efb-4201-93bd-db3389bb984d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:59.324720 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-73fdd2b6-3efb-4201-93bd-db3389bb984d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:13:59.324720 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:59.324720 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:59.324720 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:59.324720 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:59.324720 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:59.324720 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:59.324720 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:59.324720 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:59.324720 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:59.324720 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:59.324720 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:59.324720 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:13:59.324720 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:13:59.324720 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:13:59.326347 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:13:59.326347 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:59.326347 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:59.326347 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:59.326347 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:59.326347 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:59.326347 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:59.326347 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:59.326347 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:59.326347 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:59.326347 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:59.326347 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:59.326347 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:59.326347 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:59.326347 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:59.326347 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:59.326347 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:59.327267 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:13:59.327267 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:13:59.327267 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:13:59.327267 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:59.327267 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 287/558] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:13:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:59.331542 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fdd2b6-3efb-4201-93bd-db3389bb984d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:59.383790 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:59.398536 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fea0fb-c7cb-4cb2-8aba-6a88bdff8f2d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:59.409271 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:59.436604 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f6b93b-b1a6-49f3-90cc-51b4eb78819e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:59.437427 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6b93b-b1a6-49f3-90cc-51b4eb78819e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:59.437807 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6b93b-b1a6-49f3-90cc-51b4eb78819e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:59.438178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6b93b-b1a6-49f3-90cc-51b4eb78819e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:59.441600 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fdd2b6-3efb-4201-93bd-db3389bb984d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:59.446242 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f6b93b-b1a6-49f3-90cc-51b4eb78819e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:59.447877 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f6b93b-b1a6-49f3-90cc-51b4eb78819e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:59.460706 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-06f6b93b-b1a6-49f3-90cc-51b4eb78819e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 05 10:13:59.460706 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:59.460706 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:59.460706 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:59.460706 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:59.460706 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:59.460706 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:59.460706 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:59.460706 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:59.460706 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:59.460706 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:59.460706 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:59.460706 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 10:13:59.460706 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 10:13:59.460706 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 10:13:59.460706 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 10:13:59.461830 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:59.461830 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:59.461830 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 10:13:59.461830 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:13:59.461830 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 10:13:59.461830 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:13:59.461830 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:59.461830 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:59.461830 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:59.461830 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:59.461830 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:59.461830 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:59.461830 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:59.461830 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:59.461830 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:59.461830 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:59.461830 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:59.461830 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:59.463081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:59.463081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:59.463081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 10:13:59.463081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 10:13:59.463081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 10:13:59.463081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 10:13:59.463081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 10:13:59.463081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 10:13:59.463081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 05 10:13:59.463081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:13:59.464987 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 272/559] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 10:13:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:59.466099 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f6b93b-b1a6-49f3-90cc-51b4eb78819e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:59.481754 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7adf6f-a83f-4577-ab2b-b95ebe6b1fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:59.482745 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7adf6f-a83f-4577-ab2b-b95ebe6b1fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:59.483208 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7adf6f-a83f-4577-ab2b-b95ebe6b1fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:59.483708 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7adf6f-a83f-4577-ab2b-b95ebe6b1fa0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:59.495253 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7adf6f-a83f-4577-ab2b-b95ebe6b1fa0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:13:59.501287 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-4a7adf6f-a83f-4577-ab2b-b95ebe6b1fa0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 10:13:59.501287 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:59.501287 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:59.501287 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:59.501287 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:59.501287 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:59.501287 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:59.501287 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:59.501287 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:59.501287 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:59.501287 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:59.501287 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:59.501287 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:13:59.501287 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:13:59.501287 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:13:59.501287 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:13:59.502197 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:13:59.502197 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:13:59.502197 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:13:59.502197 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:13:59.502197 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:59.502197 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:59.502197 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:59.502197 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:59.502197 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:59.502197 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:59.502197 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:59.502197 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:59.502197 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:59.502197 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:59.502197 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:59.502197 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:59.503135 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:59.503135 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:59.503135 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:59.503135 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:59.503135 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:13:59.503135 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:13:59.503135 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:13:59.503135 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:13:59.503135 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:13:59.503135 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:13:59.503135 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 10:13:59.503135 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:59.505446 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 288/560] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 10:13:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:59.506424 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7adf6f-a83f-4577-ab2b-b95ebe6b1fa0 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:59.579092 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f6b93b-b1a6-49f3-90cc-51b4eb78819e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:59.608701 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7adf6f-a83f-4577-ab2b-b95ebe6b1fa0 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:59.615434 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f7b0a-b0fe-40aa-82c6-78e751a7397b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:59.616126 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f7b0a-b0fe-40aa-82c6-78e751a7397b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:59.616411 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f7b0a-b0fe-40aa-82c6-78e751a7397b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:59.616675 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f7b0a-b0fe-40aa-82c6-78e751a7397b None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:59.617500 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:59.618367 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9f7b0a-b0fe-40aa-82c6-78e751a7397b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:59.618756 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9f7b0a-b0fe-40aa-82c6-78e751a7397b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:59.620368 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9f7b0a-b0fe-40aa-82c6-78e751a7397b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:59.638675 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 273/561] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:13:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:59.639640 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f7b0a-b0fe-40aa-82c6-78e751a7397b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:59.644931 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4faecf-f4fd-4aa8-9d5a-bdbae20c11cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:59.645563 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4faecf-f4fd-4aa8-9d5a-bdbae20c11cb None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:59.645846 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4faecf-f4fd-4aa8-9d5a-bdbae20c11cb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:59.646121 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4faecf-f4fd-4aa8-9d5a-bdbae20c11cb None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:59.646934 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4faecf-f4fd-4aa8-9d5a-bdbae20c11cb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:59.647251 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4faecf-f4fd-4aa8-9d5a-bdbae20c11cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:59.648700 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4faecf-f4fd-4aa8-9d5a-bdbae20c11cb None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:59.667892 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 289/562] 10.222.0.40 () {56 vars in 1135 bytes} [Fri Aug 5 10:13:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:59.669530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbfe4c0-a5e3-4e96-b553-24b6fabfbe07 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:59.669819 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbfe4c0-a5e3-4e96-b553-24b6fabfbe07 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:59.670096 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbfe4c0-a5e3-4e96-b553-24b6fabfbe07 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:59.671489 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 290/563] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:59.672030 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbfe4c0-a5e3-4e96-b553-24b6fabfbe07 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:59.677360 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:59.755220 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f7b0a-b0fe-40aa-82c6-78e751a7397b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:59.775501 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbfe4c0-a5e3-4e96-b553-24b6fabfbe07 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:59.787874 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd526ca-ad09-4cb9-967e-4999e094fc91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:59.788486 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd526ca-ad09-4cb9-967e-4999e094fc91 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:59.788765 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd526ca-ad09-4cb9-967e-4999e094fc91 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:59.789035 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd526ca-ad09-4cb9-967e-4999e094fc91 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:59.789739 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd526ca-ad09-4cb9-967e-4999e094fc91 None admin] RBAC: Adding query filter params (name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:13:59.790039 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd526ca-ad09-4cb9-967e-4999e094fc91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:59.791446 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd526ca-ad09-4cb9-967e-4999e094fc91 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:59.799694 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 274/564] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 5 10:13:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:13:59.800549 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd526ca-ad09-4cb9-967e-4999e094fc91 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:59.810751 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae158c-650e-4b15-818f-d53aff142b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:59.811426 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae158c-650e-4b15-818f-d53aff142b0d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:59.811708 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae158c-650e-4b15-818f-d53aff142b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:59.811977 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae158c-650e-4b15-818f-d53aff142b0d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:59.819059 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ae158c-650e-4b15-818f-d53aff142b0d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:13:59.820526 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ae158c-650e-4b15-818f-d53aff142b0d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:13:59.832101 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-c0ae158c-650e-4b15-818f-d53aff142b0d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:13:59.832101 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:13:59.832101 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:13:59.832101 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:13:59.832101 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:13:59.832101 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:13:59.832101 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:13:59.832101 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:13:59.832101 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:13:59.832101 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:13:59.832101 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:13:59.832101 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:13:59.832101 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 10:13:59.832101 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 10:13:59.832101 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 10:13:59.832965 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 10:13:59.832965 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:13:59.832965 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:13:59.832965 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:13:59.832965 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:13:59.832965 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:13:59.832965 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:13:59.832965 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:13:59.832965 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:13:59.832965 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:13:59.832965 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:13:59.832965 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:13:59.832965 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:13:59.832965 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:13:59.832965 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:13:59.832965 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:13:59.832965 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:13:59.833837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 10:13:59.833837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 10:13:59.833837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 10:13:59.833837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 10:13:59.833837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 10:13:59.833837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 10:13:59.833837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:13:59.833837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:13:59.833837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 291/565] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:13:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:13:59.834491 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb45fce-2450-40f5-b1df-a064291e1db2 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:59.834703 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb45fce-2450-40f5-b1df-a064291e1db2 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:59.834955 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb45fce-2450-40f5-b1df-a064291e1db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:59.852831 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-4bb45fce-2450-40f5-b1df-a064291e1db2 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:13:59.854013 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4bb45fce-2450-40f5-b1df-a064291e1db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:13:59.875101 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:59.897824 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:13:59.908413 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd526ca-ad09-4cb9-967e-4999e094fc91 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:13:59.923583 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 292/566] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:13:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:13:59.925028 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cff762-86b8-44f3-9746-e6b5c40f8f33 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:59.925306 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cff762-86b8-44f3-9746-e6b5c40f8f33 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:59.925578 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cff762-86b8-44f3-9746-e6b5c40f8f33 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:13:59.926851 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 293/567] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:13:59.927567 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cff762-86b8-44f3-9746-e6b5c40f8f33 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:13:59.958558 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13323cf-4088-48e6-89d7-a363ee347df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:13:59.959235 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13323cf-4088-48e6-89d7-a363ee347df0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:13:59.959518 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13323cf-4088-48e6-89d7-a363ee347df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:13:59.959798 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13323cf-4088-48e6-89d7-a363ee347df0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:00.027306 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13323cf-4088-48e6-89d7-a363ee347df0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:00.028871 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13323cf-4088-48e6-89d7-a363ee347df0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:00.029822 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cff762-86b8-44f3-9746-e6b5c40f8f33 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:00.063582 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3d42b0-5470-417e-8ed9-611829603b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:00.064219 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d42b0-5470-417e-8ed9-611829603b89 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:00.064494 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d42b0-5470-417e-8ed9-611829603b89 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:00.064760 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d42b0-5470-417e-8ed9-611829603b89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:00.074535 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3d42b0-5470-417e-8ed9-611829603b89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:00.076225 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3d42b0-5470-417e-8ed9-611829603b89 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:00.091308 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-da3d42b0-5470-417e-8ed9-611829603b89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:00.091308 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:00.091308 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:00.091308 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:00.091308 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:00.091308 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:00.091308 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:00.091308 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:00.091308 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:00.091308 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:00.091308 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:00.091308 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:00.091308 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:00.091308 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:00.091308 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:00.092257 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:00.092257 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:00.092257 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:00.092257 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:00.092257 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:00.092257 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:00.092257 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:00.092257 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:00.092257 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:00.092257 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:00.092257 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:00.092257 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:00.092257 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:00.092257 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:00.092257 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:00.092257 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:00.092257 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:00.093148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:00.093148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:00.093148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:00.093148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:00.093148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 294/568] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:13:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:00.093427 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3d42b0-5470-417e-8ed9-611829603b89 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:00.104009 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:00.122558 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:00.133565 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 275/569] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:13:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 10:14:00.134856 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933ad8c-e487-4fa2-8408-f1b1af798a87 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:00.135166 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933ad8c-e487-4fa2-8408-f1b1af798a87 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:00.135437 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933ad8c-e487-4fa2-8408-f1b1af798a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:00.152665 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-8933ad8c-e487-4fa2-8408-f1b1af798a87 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:00.153951 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8933ad8c-e487-4fa2-8408-f1b1af798a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:00.200803 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3d42b0-5470-417e-8ed9-611829603b89 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:00.219437 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 276/570] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:14:00.220270 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8933ad8c-e487-4fa2-8408-f1b1af798a87 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:00.244012 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd70bb-d0ca-4547-af0f-71488a35cd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:00.244639 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd70bb-d0ca-4547-af0f-71488a35cd45 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:00.244915 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd70bb-d0ca-4547-af0f-71488a35cd45 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:00.245185 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd70bb-d0ca-4547-af0f-71488a35cd45 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:00.245933 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdd70bb-d0ca-4547-af0f-71488a35cd45 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:00.246236 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdd70bb-d0ca-4547-af0f-71488a35cd45 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:00.248066 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdd70bb-d0ca-4547-af0f-71488a35cd45 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:00.260517 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 295/571] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 10:14:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:00.261330 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd70bb-d0ca-4547-af0f-71488a35cd45 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:00.326077 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8933ad8c-e487-4fa2-8408-f1b1af798a87 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:00.327198 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:00.346733 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:00.360454 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd70bb-d0ca-4547-af0f-71488a35cd45 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:00.361278 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d135bd-2031-42df-aa4e-02a6f6fe9dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:00.361927 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d135bd-2031-42df-aa4e-02a6f6fe9dc0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:00.362203 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d135bd-2031-42df-aa4e-02a6f6fe9dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:00.362481 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d135bd-2031-42df-aa4e-02a6f6fe9dc0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:00.372973 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d135bd-2031-42df-aa4e-02a6f6fe9dc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:14:00.377868 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-26d135bd-2031-42df-aa4e-02a6f6fe9dc0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 10:14:00.377868 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:00.377868 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:00.377868 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:00.377868 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:00.377868 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:00.377868 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:00.377868 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:00.377868 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:00.377868 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:00.377868 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:00.377868 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:00.377868 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:14:00.377868 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:14:00.377868 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:14:00.377868 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:14:00.379037 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:14:00.379037 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:14:00.379037 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:14:00.379037 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:14:00.379037 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:00.379037 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:00.379037 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:00.379037 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:00.379037 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:00.379037 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:00.379037 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:00.379037 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:00.379037 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:00.379037 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:00.379037 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:00.379037 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:00.379898 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:00.379898 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:00.379898 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:00.379898 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:00.379898 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:14:00.379898 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:14:00.379898 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:14:00.379898 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:14:00.379898 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:14:00.379898 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:14:00.379898 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 10:14:00.379898 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:00.379898 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 277/572] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 10:14:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:00.379898 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d135bd-2031-42df-aa4e-02a6f6fe9dc0 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:00.397648 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f01094-3d22-44f3-b65b-3a76157b44d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:00.398266 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f01094-3d22-44f3-b65b-3a76157b44d4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:00.398542 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f01094-3d22-44f3-b65b-3a76157b44d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:00.398807 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f01094-3d22-44f3-b65b-3a76157b44d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:00.399530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f01094-3d22-44f3-b65b-3a76157b44d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:00.399828 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f01094-3d22-44f3-b65b-3a76157b44d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:00.401206 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f01094-3d22-44f3-b65b-3a76157b44d4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:00.412714 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 296/573] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:00.414888 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f01094-3d22-44f3-b65b-3a76157b44d4 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:00.482515 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d135bd-2031-42df-aa4e-02a6f6fe9dc0 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:00.512892 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f01094-3d22-44f3-b65b-3a76157b44d4 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:00.515538 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e846155-8443-4527-8a7c-23a6e8505384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:00.516181 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e846155-8443-4527-8a7c-23a6e8505384 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:00.516461 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e846155-8443-4527-8a7c-23a6e8505384 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:00.516735 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e846155-8443-4527-8a7c-23a6e8505384 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:00.527165 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e846155-8443-4527-8a7c-23a6e8505384 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:00.528633 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e846155-8443-4527-8a7c-23a6e8505384 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:00.538154 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:00.542556 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-0e846155-8443-4527-8a7c-23a6e8505384 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:00.542556 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:00.542556 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:00.542556 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:00.542556 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:00.542556 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:00.542556 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:00.542556 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:00.542556 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:00.542556 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:00.542556 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:00.542556 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:00.542556 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:00.542556 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:00.542556 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:00.544242 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:00.544242 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:00.544242 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:00.544242 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:00.544242 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:00.544242 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:00.544242 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:00.544242 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:00.544242 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:00.544242 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:00.544242 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:00.544242 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:00.544242 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:00.544242 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:00.544242 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:00.544242 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:00.544242 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:00.545483 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:00.545483 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:00.545483 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:00.545483 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:00.545483 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 278/574] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:00.545483 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e846155-8443-4527-8a7c-23a6e8505384 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:00.549848 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d13ad54-747e-4687-8814-f0a0d723a76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:00.550479 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d13ad54-747e-4687-8814-f0a0d723a76d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:00.550759 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d13ad54-747e-4687-8814-f0a0d723a76d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:00.551071 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d13ad54-747e-4687-8814-f0a0d723a76d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:00.562233 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d13ad54-747e-4687-8814-f0a0d723a76d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:14:00.567549 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-9d13ad54-747e-4687-8814-f0a0d723a76d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 10:14:00.567549 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:00.567549 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:00.567549 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:00.567549 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:00.567549 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:00.567549 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:00.567549 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:00.567549 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:00.567549 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:00.567549 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:00.567549 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:00.567549 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:14:00.567549 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:14:00.567549 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:14:00.567549 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:14:00.568485 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:14:00.568485 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:14:00.568485 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:14:00.568485 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:14:00.568485 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:00.568485 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:00.568485 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:00.568485 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:00.568485 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:00.568485 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:00.568485 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:00.568485 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:00.568485 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:00.568485 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:00.568485 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:00.568485 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:00.569288 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:00.569288 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:00.569288 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:00.569288 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:00.569288 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:14:00.569288 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:14:00.569288 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:14:00.569288 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:14:00.569288 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:14:00.569288 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:14:00.569288 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 10:14:00.569288 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:00.569288 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 297/575] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 10:14:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:00.569968 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d13ad54-747e-4687-8814-f0a0d723a76d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:00.580600 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:00.648467 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e846155-8443-4527-8a7c-23a6e8505384 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:00.669799 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d13ad54-747e-4687-8814-f0a0d723a76d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:00.682357 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7260718f-7357-478d-9436-fed1f1458438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:00.683069 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260718f-7357-478d-9436-fed1f1458438 None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:00.683350 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260718f-7357-478d-9436-fed1f1458438 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:00.683622 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260718f-7357-478d-9436-fed1f1458438 None admin] PATH_INFO: `/v3/projects/36b8b505012d4d0bac54845b6799e743/users/ac0a8cf7f8b74e88a04444ec38547528/roles/3a2b4319820a488198dabee34420cdc7` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:00.698560 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7260718f-7357-478d-9436-fed1f1458438 None admin] RBAC: Authorizing `identity:create_grant(project_id=36b8b505012d4d0bac54845b6799e743, user_id=ac0a8cf7f8b74e88a04444ec38547528, role_id=3a2b4319820a488198dabee34420cdc7)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:00.700029 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7260718f-7357-478d-9436-fed1f1458438 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:00.703178 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72beb079-65d7-4d75-9218-6348906982a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:00.703804 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72beb079-65d7-4d75-9218-6348906982a2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:00.704253 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72beb079-65d7-4d75-9218-6348906982a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:00.704461 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72beb079-65d7-4d75-9218-6348906982a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:00.705061 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72beb079-65d7-4d75-9218-6348906982a2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:00.705369 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72beb079-65d7-4d75-9218-6348906982a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:00.706737 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72beb079-65d7-4d75-9218-6348906982a2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:00.714155 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 279/576] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 10:14:00 2022] PUT /identity/v3/projects/36b8b505012d4d0bac54845b6799e743/users/ac0a8cf7f8b74e88a04444ec38547528/roles/3a2b4319820a488198dabee34420cdc7 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:14:00.719578 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7260718f-7357-478d-9436-fed1f1458438 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:00.719800 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 298/577] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 5 10:14:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:00.719800 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72beb079-65d7-4d75-9218-6348906982a2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:00.781354 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:00.815799 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:00.838790 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7260718f-7357-478d-9436-fed1f1458438 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:00.846026 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72beb079-65d7-4d75-9218-6348906982a2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:00.881755 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c5753b-3274-4996-a969-36d14d383934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:00.882417 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5753b-3274-4996-a969-36d14d383934 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:00.882701 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5753b-3274-4996-a969-36d14d383934 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:00.883011 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5753b-3274-4996-a969-36d14d383934 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:00.889348 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576aa59e-6531-4b8c-ac16-f0e938262d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:00.889979 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576aa59e-6531-4b8c-ac16-f0e938262d2e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:00.890260 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576aa59e-6531-4b8c-ac16-f0e938262d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:00.890539 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576aa59e-6531-4b8c-ac16-f0e938262d2e None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:00.891260 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576aa59e-6531-4b8c-ac16-f0e938262d2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:00.891566 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576aa59e-6531-4b8c-ac16-f0e938262d2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:00.891664 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c5753b-3274-4996-a969-36d14d383934 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:00.892970 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576aa59e-6531-4b8c-ac16-f0e938262d2e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:00.893078 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c5753b-3274-4996-a969-36d14d383934 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:00.905224 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 299/578] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:00.906066 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576aa59e-6531-4b8c-ac16-f0e938262d2e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:00.907619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-58c5753b-3274-4996-a969-36d14d383934 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 10:14:00.907619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:00.907619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:00.907619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:00.907619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:00.907619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:00.907619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:00.907619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:00.907619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:00.907619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:00.907619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:00.907619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:00.907619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 10:14:00.907619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 10:14:00.907619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 10:14:00.907619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 10:14:00.908697 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:00.908697 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:00.908697 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 10:14:00.908697 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:00.908697 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 10:14:00.908697 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:00.908697 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:00.908697 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:00.908697 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:00.908697 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:00.908697 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:00.908697 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:00.908697 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:00.908697 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:00.908697 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:00.908697 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:00.908697 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:00.908697 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:00.909766 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:00.909766 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:00.909766 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 10:14:00.909766 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 10:14:00.909766 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 10:14:00.909766 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 10:14:00.909766 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 10:14:00.909766 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 10:14:00.909766 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 10:14:00.909766 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:00.912096 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 280/579] 10.222.0.40 () {56 vars in 1093 bytes} [Fri Aug 5 10:14:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:00.913186 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c5753b-3274-4996-a969-36d14d383934 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:01.008196 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576aa59e-6531-4b8c-ac16-f0e938262d2e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:01.018088 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c5753b-3274-4996-a969-36d14d383934 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:01.019647 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:01.046659 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304cbd65-1299-42ab-a142-27bfe3ba25f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:01.047326 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304cbd65-1299-42ab-a142-27bfe3ba25f0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:01.047607 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304cbd65-1299-42ab-a142-27bfe3ba25f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:01.047879 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304cbd65-1299-42ab-a142-27bfe3ba25f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:01.048595 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304cbd65-1299-42ab-a142-27bfe3ba25f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:01.048900 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304cbd65-1299-42ab-a142-27bfe3ba25f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:01.050274 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304cbd65-1299-42ab-a142-27bfe3ba25f0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:01.053940 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba9c190-3eab-46fc-8222-29348084f112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:01.054580 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba9c190-3eab-46fc-8222-29348084f112 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:01.054857 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba9c190-3eab-46fc-8222-29348084f112 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:01.055180 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba9c190-3eab-46fc-8222-29348084f112 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:01.063398 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 300/580] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 5 10:14:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:01.063398 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304cbd65-1299-42ab-a142-27bfe3ba25f0 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:01.067768 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba9c190-3eab-46fc-8222-29348084f112 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:01.069285 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba9c190-3eab-46fc-8222-29348084f112 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:01.071287 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:01.085164 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-5ba9c190-3eab-46fc-8222-29348084f112 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:01.085164 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:01.085164 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:01.085164 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:01.085164 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:01.085164 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:01.085164 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:01.085164 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:01.085164 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:01.085164 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:01.085164 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:01.085164 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:01.085164 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:01.085164 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:01.085164 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:01.086081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:01.086081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:01.086081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:01.086081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:01.086081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:01.086081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:01.086081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:01.086081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:01.086081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:01.086081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:01.086081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:01.086081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:01.086081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:01.086081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:01.086081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:01.086081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:01.086081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:01.087118 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:01.087118 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:01.087118 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:01.087118 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:01.087118 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 281/581] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:01.087118 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba9c190-3eab-46fc-8222-29348084f112 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:01.177731 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304cbd65-1299-42ab-a142-27bfe3ba25f0 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:01.194455 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba9c190-3eab-46fc-8222-29348084f112 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:01.216671 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb084c-1bc4-4c6f-a69c-a0f3845f44f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:01.217614 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb084c-1bc4-4c6f-a69c-a0f3845f44f4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:01.217896 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb084c-1bc4-4c6f-a69c-a0f3845f44f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:01.218305 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb084c-1bc4-4c6f-a69c-a0f3845f44f4 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:01.219159 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deb084c-1bc4-4c6f-a69c-a0f3845f44f4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:01.219469 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deb084c-1bc4-4c6f-a69c-a0f3845f44f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:01.220903 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deb084c-1bc4-4c6f-a69c-a0f3845f44f4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:01.229132 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36cfbdd-2e97-4f79-a828-0ae08bca33a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:01.229694 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36cfbdd-2e97-4f79-a828-0ae08bca33a0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:01.229973 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36cfbdd-2e97-4f79-a828-0ae08bca33a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:01.230925 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36cfbdd-2e97-4f79-a828-0ae08bca33a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:01.237165 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36cfbdd-2e97-4f79-a828-0ae08bca33a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:01.238608 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36cfbdd-2e97-4f79-a828-0ae08bca33a0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:01.239139 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 301/582] 10.222.0.40 () {56 vars in 1133 bytes} [Fri Aug 5 10:14:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:01.240012 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb084c-1bc4-4c6f-a69c-a0f3845f44f4 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:01.254974 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-a36cfbdd-2e97-4f79-a828-0ae08bca33a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:01.254974 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:01.254974 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:01.254974 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:01.254974 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:01.254974 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:01.254974 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:01.254974 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:01.254974 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:01.254974 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:01.254974 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:01.254974 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:01.254974 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 10:14:01.254974 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 10:14:01.254974 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 10:14:01.255876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 10:14:01.255876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:01.255876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:01.255876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:01.255876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:01.255876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:01.255876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:01.255876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:01.255876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:01.255876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:01.255876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:01.255876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:01.255876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:01.255876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:01.255876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:01.255876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:01.255876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:01.256767 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 10:14:01.256767 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 10:14:01.256767 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 10:14:01.256767 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 10:14:01.256767 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 10:14:01.256767 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 10:14:01.256767 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:01.256767 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:01.256767 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 282/583] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:14:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:01.257246 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36cfbdd-2e97-4f79-a828-0ae08bca33a0 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:01.275644 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:01.314729 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:01.348962 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb084c-1bc4-4c6f-a69c-a0f3845f44f4 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:01.364524 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36cfbdd-2e97-4f79-a828-0ae08bca33a0 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:01.384374 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf323f-24c6-4318-98e2-ec968b9aa324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:01.385018 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf323f-24c6-4318-98e2-ec968b9aa324 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:01.385293 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf323f-24c6-4318-98e2-ec968b9aa324 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:01.385572 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf323f-24c6-4318-98e2-ec968b9aa324 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:01.396464 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bf323f-24c6-4318-98e2-ec968b9aa324 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:01.398255 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bf323f-24c6-4318-98e2-ec968b9aa324 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:01.399414 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acbe651-15e6-42ad-9f54-97b3edf41106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:01.400040 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acbe651-15e6-42ad-9f54-97b3edf41106 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:01.400317 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acbe651-15e6-42ad-9f54-97b3edf41106 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:01.400589 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acbe651-15e6-42ad-9f54-97b3edf41106 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:01.401287 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acbe651-15e6-42ad-9f54-97b3edf41106 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:01.401581 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acbe651-15e6-42ad-9f54-97b3edf41106 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:01.402960 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acbe651-15e6-42ad-9f54-97b3edf41106 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:01.415152 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 283/584] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:01.415900 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acbe651-15e6-42ad-9f54-97b3edf41106 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:01.419388 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-c9bf323f-24c6-4318-98e2-ec968b9aa324 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:01.419388 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:01.419388 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:01.419388 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:01.419388 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:01.419388 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:01.419388 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:01.419388 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:01.419388 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:01.419388 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:01.419388 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:01.419388 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:01.419388 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:01.419388 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:01.419388 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:01.420275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:01.420275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:01.420275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:01.420275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:01.420275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:01.420275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:01.420275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:01.420275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:01.420275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:01.420275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:01.420275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:01.420275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:01.420275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:01.420275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:01.420275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:01.420275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:01.420275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:01.421152 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:01.421152 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:01.421152 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:01.421152 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:01.421152 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 302/585] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:01.421591 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf323f-24c6-4318-98e2-ec968b9aa324 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:01.511412 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:01.522801 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acbe651-15e6-42ad-9f54-97b3edf41106 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:01.530825 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf323f-24c6-4318-98e2-ec968b9aa324 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:01.537198 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:01.559604 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd53c9e-6ef4-4144-9c2d-d8686ae9d9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:01.560311 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd53c9e-6ef4-4144-9c2d-d8686ae9d9fe None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:01.560590 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd53c9e-6ef4-4144-9c2d-d8686ae9d9fe None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:01.560864 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd53c9e-6ef4-4144-9c2d-d8686ae9d9fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:01.568435 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ace227f-9c4f-4417-8954-e95c34c12314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:01.569296 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace227f-9c4f-4417-8954-e95c34c12314 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:01.569646 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace227f-9c4f-4417-8954-e95c34c12314 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:01.569923 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace227f-9c4f-4417-8954-e95c34c12314 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:01.570675 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ace227f-9c4f-4417-8954-e95c34c12314 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:01.571107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd53c9e-6ef4-4144-9c2d-d8686ae9d9fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:01.571299 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ace227f-9c4f-4417-8954-e95c34c12314 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:01.572550 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd53c9e-6ef4-4144-9c2d-d8686ae9d9fe None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:01.572741 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ace227f-9c4f-4417-8954-e95c34c12314 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:01.589992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 303/586] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 10:14:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:01.589992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ace227f-9c4f-4417-8954-e95c34c12314 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:01.590732 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-bdd53c9e-6ef4-4144-9c2d-d8686ae9d9fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:01.590732 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:01.590732 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:01.590732 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:01.590732 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:01.590732 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:01.590732 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:01.590732 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:01.590732 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:01.590732 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:01.590732 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:01.590732 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:01.590732 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:01.590732 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:01.590732 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:01.591590 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:01.591590 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:01.591590 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:01.591590 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:01.591590 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:01.591590 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:01.591590 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:01.591590 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:01.591590 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:01.591590 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:01.591590 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:01.591590 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:01.591590 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:01.591590 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:01.591590 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:01.591590 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:01.591590 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:01.592476 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:01.592476 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:01.592476 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:01.592476 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:01.592476 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 284/587] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:01.592476 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd53c9e-6ef4-4144-9c2d-d8686ae9d9fe None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:01.691765 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd53c9e-6ef4-4144-9c2d-d8686ae9d9fe None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:01.694929 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ace227f-9c4f-4417-8954-e95c34c12314 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:01.725847 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9563e33-9046-4bff-8213-907dca9a1971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:01.726752 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9563e33-9046-4bff-8213-907dca9a1971 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:01.727246 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9563e33-9046-4bff-8213-907dca9a1971 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:01.727522 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9563e33-9046-4bff-8213-907dca9a1971 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:01.728248 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9563e33-9046-4bff-8213-907dca9a1971 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:01.728554 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9563e33-9046-4bff-8213-907dca9a1971 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:01.729940 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9563e33-9046-4bff-8213-907dca9a1971 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:01.731835 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddf9ab3-13f9-4ac5-9ea0-f901a71a2abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:01.732903 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf9ab3-13f9-4ac5-9ea0-f901a71a2abd None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:01.733183 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf9ab3-13f9-4ac5-9ea0-f901a71a2abd None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:01.733455 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf9ab3-13f9-4ac5-9ea0-f901a71a2abd None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:01.746025 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 285/588] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:01.746206 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddf9ab3-13f9-4ac5-9ea0-f901a71a2abd None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:01.746308 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9563e33-9046-4bff-8213-907dca9a1971 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:01.746414 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddf9ab3-13f9-4ac5-9ea0-f901a71a2abd None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:01.758604 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-dddf9ab3-13f9-4ac5-9ea0-f901a71a2abd None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 10:14:01.758604 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:01.758604 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:01.758604 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:01.758604 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:01.758604 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:01.758604 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:01.758604 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:01.758604 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:01.758604 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:01.758604 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:01.758604 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:01.758604 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 10:14:01.758604 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 10:14:01.758604 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 10:14:01.758604 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 10:14:01.759606 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:01.759606 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:01.759606 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 10:14:01.759606 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:01.759606 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 10:14:01.759606 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:01.759606 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:01.759606 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:01.759606 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:01.759606 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:01.759606 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:01.759606 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:01.759606 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:01.759606 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:01.759606 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:01.759606 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:01.759606 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:01.759606 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:01.760530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:01.760530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:01.760530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 10:14:01.760530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 10:14:01.760530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 10:14:01.760530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 10:14:01.760530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 10:14:01.760530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 10:14:01.760530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 10:14:01.760530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:01.760530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 304/589] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:14:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:01.760530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddf9ab3-13f9-4ac5-9ea0-f901a71a2abd None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:01.765387 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:01.782249 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:01.862586 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9563e33-9046-4bff-8213-907dca9a1971 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:01.873123 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddf9ab3-13f9-4ac5-9ea0-f901a71a2abd None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:01.898208 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96f7905-2c81-479e-9200-6d857bdd1308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:01.898824 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f7905-2c81-479e-9200-6d857bdd1308 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:01.899139 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f7905-2c81-479e-9200-6d857bdd1308 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:01.899406 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f7905-2c81-479e-9200-6d857bdd1308 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:01.900193 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96f7905-2c81-479e-9200-6d857bdd1308 None admin] RBAC: Adding query filter params (role.id=3a2b4319820a488198dabee34420cdc7, scope.project.id=36b8b505012d4d0bac54845b6799e743, user.id=c80cf0e429f04f229aea1aa9293a93b2) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:01.900500 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96f7905-2c81-479e-9200-6d857bdd1308 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:01.901929 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96f7905-2c81-479e-9200-6d857bdd1308 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:01.907869 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 286/590] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:14:01 2022] GET /identity/v3/role_assignments?user.id=c80cf0e429f04f229aea1aa9293a93b2&scope.project.id=36b8b505012d4d0bac54845b6799e743&role.id=3a2b4319820a488198dabee34420cdc7 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:01.908651 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96f7905-2c81-479e-9200-6d857bdd1308 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:01.921061 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa10c0-2ee8-43d7-8d0c-5ed382693ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:01.921809 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa10c0-2ee8-43d7-8d0c-5ed382693ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:01.922162 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa10c0-2ee8-43d7-8d0c-5ed382693ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:01.922505 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa10c0-2ee8-43d7-8d0c-5ed382693ca2 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:01.923318 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaa10c0-2ee8-43d7-8d0c-5ed382693ca2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:01.923701 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaa10c0-2ee8-43d7-8d0c-5ed382693ca2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:01.925277 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaa10c0-2ee8-43d7-8d0c-5ed382693ca2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:01.945305 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 305/591] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:01.946326 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa10c0-2ee8-43d7-8d0c-5ed382693ca2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:02.022262 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:02.028319 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:02.039637 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96f7905-2c81-479e-9200-6d857bdd1308 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:02.076209 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa10c0-2ee8-43d7-8d0c-5ed382693ca2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:02.080186 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa6b57c-ffd7-410e-bab5-2e0e03392ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:02.080843 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6b57c-ffd7-410e-bab5-2e0e03392ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:02.081123 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6b57c-ffd7-410e-bab5-2e0e03392ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:02.081391 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6b57c-ffd7-410e-bab5-2e0e03392ef5 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:02.090067 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa6b57c-ffd7-410e-bab5-2e0e03392ef5 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:02.091888 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa6b57c-ffd7-410e-bab5-2e0e03392ef5 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:02.105412 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-9fa6b57c-ffd7-410e-bab5-2e0e03392ef5 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 05 10:14:02.105412 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:02.105412 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:02.105412 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:02.105412 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:02.105412 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:02.105412 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:02.105412 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:02.105412 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:02.105412 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:02.105412 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:02.105412 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:02.105412 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 10:14:02.105412 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 10:14:02.105412 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 10:14:02.105412 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 10:14:02.106459 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:02.106459 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:02.106459 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 10:14:02.106459 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:02.106459 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 10:14:02.106459 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:02.106459 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:02.106459 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:02.106459 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:02.106459 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:02.106459 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:02.106459 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:02.106459 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:02.106459 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:02.106459 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:02.106459 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:02.106459 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:02.106459 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:02.107433 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:02.107433 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:02.107433 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 10:14:02.107433 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 10:14:02.107433 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 10:14:02.107433 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 10:14:02.107433 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 10:14:02.107433 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 10:14:02.107433 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 05 10:14:02.107433 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:02.111637 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 287/592] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 10:14:01 2022] GET /identity/v3/users/neutron => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:02.112506 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa6b57c-ffd7-410e-bab5-2e0e03392ef5 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:02.129088 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27877b0e-403e-40a0-8cf1-972c3cfcfd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:02.129799 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27877b0e-403e-40a0-8cf1-972c3cfcfd75 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:02.130174 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27877b0e-403e-40a0-8cf1-972c3cfcfd75 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:02.130449 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27877b0e-403e-40a0-8cf1-972c3cfcfd75 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:02.131276 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27877b0e-403e-40a0-8cf1-972c3cfcfd75 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:02.131579 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27877b0e-403e-40a0-8cf1-972c3cfcfd75 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:02.133126 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27877b0e-403e-40a0-8cf1-972c3cfcfd75 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:02.153005 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 306/593] 10.222.0.40 () {56 vars in 1137 bytes} [Fri Aug 5 10:14:01 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:02.156937 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27877b0e-403e-40a0-8cf1-972c3cfcfd75 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:02.223906 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa6b57c-ffd7-410e-bab5-2e0e03392ef5 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:02.260093 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63453a33-3fe9-4d6d-ae7f-5505ca255b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:02.260991 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63453a33-3fe9-4d6d-ae7f-5505ca255b2c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:02.261432 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63453a33-3fe9-4d6d-ae7f-5505ca255b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:02.261822 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27877b0e-403e-40a0-8cf1-972c3cfcfd75 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:02.262006 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63453a33-3fe9-4d6d-ae7f-5505ca255b2c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:02.269863 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63453a33-3fe9-4d6d-ae7f-5505ca255b2c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:02.271660 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63453a33-3fe9-4d6d-ae7f-5505ca255b2c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:02.283131 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:02.284906 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-63453a33-3fe9-4d6d-ae7f-5505ca255b2c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:02.284906 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:02.284906 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:02.284906 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:02.284906 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:02.284906 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:02.284906 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:02.284906 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:02.284906 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:02.284906 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:02.284906 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:02.284906 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:02.284906 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 10:14:02.284906 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 10:14:02.284906 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 10:14:02.285933 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 10:14:02.285933 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:02.285933 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:02.285933 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:02.285933 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:02.285933 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:02.285933 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:02.285933 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:02.285933 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:02.285933 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:02.285933 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:02.285933 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:02.285933 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:02.285933 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:02.285933 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:02.285933 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:02.285933 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:02.287030 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 10:14:02.287030 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 10:14:02.287030 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 10:14:02.287030 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 10:14:02.287030 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 10:14:02.287030 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 10:14:02.287030 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:02.287030 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:02.289109 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 288/594] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:14:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:02.290169 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63453a33-3fe9-4d6d-ae7f-5505ca255b2c None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:02.298693 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b948169-2925-470c-8263-d2aed0edc493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:02.299348 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b948169-2925-470c-8263-d2aed0edc493 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:02.299622 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b948169-2925-470c-8263-d2aed0edc493 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:02.299890 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b948169-2925-470c-8263-d2aed0edc493 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:02.300685 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b948169-2925-470c-8263-d2aed0edc493 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:02.300989 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b948169-2925-470c-8263-d2aed0edc493 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:02.302522 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b948169-2925-470c-8263-d2aed0edc493 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:02.306352 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:02.328407 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 307/595] 10.222.0.40 () {56 vars in 1139 bytes} [Fri Aug 5 10:14:02 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:02.329317 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b948169-2925-470c-8263-d2aed0edc493 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:02.400325 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63453a33-3fe9-4d6d-ae7f-5505ca255b2c None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:02.436656 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b948169-2925-470c-8263-d2aed0edc493 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:02.437089 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb82e5-c122-45cf-b506-551044ebb99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:02.437888 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb82e5-c122-45cf-b506-551044ebb99b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:02.438376 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb82e5-c122-45cf-b506-551044ebb99b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:02.438875 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb82e5-c122-45cf-b506-551044ebb99b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:02.450026 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdb82e5-c122-45cf-b506-551044ebb99b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:02.452052 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdb82e5-c122-45cf-b506-551044ebb99b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:02.471691 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5918877-80e5-443d-b201-42a80e8350e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:02.472312 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5918877-80e5-443d-b201-42a80e8350e6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:02.472586 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5918877-80e5-443d-b201-42a80e8350e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:02.472822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-bfdb82e5-c122-45cf-b506-551044ebb99b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:02.472822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:02.472822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:02.472822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:02.472822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:02.472822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:02.472822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:02.472822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:02.472822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:02.472822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:02.472822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:02.472822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:02.472822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:02.472822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:02.472822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:02.473813 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:02.473813 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:02.473813 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:02.473813 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:02.473813 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:02.473813 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:02.473813 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:02.473813 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:02.473813 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:02.473813 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:02.473813 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:02.473813 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:02.473813 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:02.473813 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:02.473813 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:02.473813 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:02.473813 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:02.474990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:02.474990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:02.474990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:02.474990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:02.475291 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5918877-80e5-443d-b201-42a80e8350e6 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:02.475291 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5918877-80e5-443d-b201-42a80e8350e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:02.475291 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5918877-80e5-443d-b201-42a80e8350e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:02.475507 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5918877-80e5-443d-b201-42a80e8350e6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:02.477524 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 289/596] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:02.478053 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb82e5-c122-45cf-b506-551044ebb99b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:02.490999 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 308/597] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 10:14:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:02.491879 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5918877-80e5-443d-b201-42a80e8350e6 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:02.542723 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:02.558790 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:02.609783 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb82e5-c122-45cf-b506-551044ebb99b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:02.614512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5918877-80e5-443d-b201-42a80e8350e6 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:02.646567 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7a76ee-53dd-45ce-bc02-60ff822ec013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:02.647564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7a76ee-53dd-45ce-bc02-60ff822ec013 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:02.648037 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7a76ee-53dd-45ce-bc02-60ff822ec013 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:02.648409 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7a76ee-53dd-45ce-bc02-60ff822ec013 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:02.650753 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a093ea96-b0a2-4e06-94bc-738dabc8cde3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:02.651662 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093ea96-b0a2-4e06-94bc-738dabc8cde3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:02.652072 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093ea96-b0a2-4e06-94bc-738dabc8cde3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:02.652478 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093ea96-b0a2-4e06-94bc-738dabc8cde3 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:02.653478 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a093ea96-b0a2-4e06-94bc-738dabc8cde3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:02.653940 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a093ea96-b0a2-4e06-94bc-738dabc8cde3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:02.655492 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a093ea96-b0a2-4e06-94bc-738dabc8cde3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:02.661378 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7a76ee-53dd-45ce-bc02-60ff822ec013 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:02.663016 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7a76ee-53dd-45ce-bc02-60ff822ec013 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:02.669785 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 309/598] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:02.671057 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a093ea96-b0a2-4e06-94bc-738dabc8cde3 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:02.680107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-6d7a76ee-53dd-45ce-bc02-60ff822ec013 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:02.680107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:02.680107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:02.680107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:02.680107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:02.680107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:02.680107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:02.680107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:02.680107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:02.680107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:02.680107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:02.680107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:02.680107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:02.680107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:02.680107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:02.681221 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:02.681221 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:02.681221 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:02.681221 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:02.681221 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:02.681221 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:02.681221 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:02.681221 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:02.681221 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:02.681221 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:02.681221 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:02.681221 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:02.681221 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:02.681221 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:02.681221 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:02.681221 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:02.681221 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:02.682284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:02.682284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:02.682284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:02.682284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:02.687875 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 290/599] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:02.687875 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7a76ee-53dd-45ce-bc02-60ff822ec013 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:02.780154 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:02.783925 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a093ea96-b0a2-4e06-94bc-738dabc8cde3 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:02.795792 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7a76ee-53dd-45ce-bc02-60ff822ec013 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:02.810827 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:02.820111 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98675165-c965-4b1c-880e-3f1a16e100fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:02.820843 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98675165-c965-4b1c-880e-3f1a16e100fa None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:02.821200 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98675165-c965-4b1c-880e-3f1a16e100fa None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:02.821514 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98675165-c965-4b1c-880e-3f1a16e100fa None admin] PATH_INFO: `/v3/projects/36b8b505012d4d0bac54845b6799e743/users/b938f1f41db94b0eafe95bc8eeaae175/roles/3a2b4319820a488198dabee34420cdc7` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:02.833488 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bae4d3-f12f-458d-8617-eed4887ea54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:02.834201 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bae4d3-f12f-458d-8617-eed4887ea54a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:02.834561 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bae4d3-f12f-458d-8617-eed4887ea54a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:02.835462 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bae4d3-f12f-458d-8617-eed4887ea54a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:02.836391 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98675165-c965-4b1c-880e-3f1a16e100fa None admin] RBAC: Authorizing `identity:create_grant(project_id=36b8b505012d4d0bac54845b6799e743, user_id=b938f1f41db94b0eafe95bc8eeaae175, role_id=3a2b4319820a488198dabee34420cdc7)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:02.837847 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98675165-c965-4b1c-880e-3f1a16e100fa None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:02.844338 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bae4d3-f12f-458d-8617-eed4887ea54a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:02.845861 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bae4d3-f12f-458d-8617-eed4887ea54a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:02.852457 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 310/600] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 10:14:02 2022] PUT /identity/v3/projects/36b8b505012d4d0bac54845b6799e743/users/b938f1f41db94b0eafe95bc8eeaae175/roles/3a2b4319820a488198dabee34420cdc7 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:14:02.853130 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98675165-c965-4b1c-880e-3f1a16e100fa None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:02.866363 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-66bae4d3-f12f-458d-8617-eed4887ea54a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:02.866363 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:02.866363 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:02.866363 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:02.866363 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:02.866363 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:02.866363 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:02.866363 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:02.866363 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:02.866363 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:02.866363 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:02.866363 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:02.866363 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 10:14:02.866363 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 10:14:02.866363 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 10:14:02.867923 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 10:14:02.867923 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:02.867923 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:02.867923 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:02.867923 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:02.867923 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:02.867923 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:02.867923 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:02.867923 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:02.867923 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:02.867923 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:02.867923 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:02.867923 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:02.867923 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:02.867923 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:02.867923 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:02.867923 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:02.869002 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 10:14:02.869002 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 10:14:02.869002 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 10:14:02.869002 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 10:14:02.869002 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 10:14:02.869002 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 10:14:02.869002 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:02.869002 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:02.877397 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 291/601] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:14:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:02.877397 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824efb4-47e3-4a07-ace0-7698677b6f82 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:02.877397 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824efb4-47e3-4a07-ace0-7698677b6f82 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:02.877397 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824efb4-47e3-4a07-ace0-7698677b6f82 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:02.879062 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 292/602] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:02.880321 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1824efb4-47e3-4a07-ace0-7698677b6f82 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:02.991145 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98675165-c965-4b1c-880e-3f1a16e100fa None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:03.033958 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1824efb4-47e3-4a07-ace0-7698677b6f82 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:03.034247 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f6be3-df90-40c9-bd7e-7588229e692e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:03.034867 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f6be3-df90-40c9-bd7e-7588229e692e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:03.035180 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f6be3-df90-40c9-bd7e-7588229e692e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:03.035454 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f6be3-df90-40c9-bd7e-7588229e692e None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:03.036158 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f6be3-df90-40c9-bd7e-7588229e692e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:03.036463 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f6be3-df90-40c9-bd7e-7588229e692e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:03.038971 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f6be3-df90-40c9-bd7e-7588229e692e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:03.051689 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 311/603] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:03.053275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929da49-4311-4cf7-83dd-421ec4b787ed None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:03.053630 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929da49-4311-4cf7-83dd-421ec4b787ed None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:03.053904 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929da49-4311-4cf7-83dd-421ec4b787ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:03.070560 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99415bad-810d-4a50-a001-daa45bff73c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:03.071389 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99415bad-810d-4a50-a001-daa45bff73c3 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:03.071758 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99415bad-810d-4a50-a001-daa45bff73c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:03.072120 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99415bad-810d-4a50-a001-daa45bff73c3 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:03.072962 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99415bad-810d-4a50-a001-daa45bff73c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:03.073359 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99415bad-810d-4a50-a001-daa45bff73c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:03.074892 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99415bad-810d-4a50-a001-daa45bff73c3 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:03.075451 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6929da49-4311-4cf7-83dd-421ec4b787ed None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:03.076990 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6929da49-4311-4cf7-83dd-421ec4b787ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:03.086923 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:03.088166 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 293/604] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 10:14:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:03.089103 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99415bad-810d-4a50-a001-daa45bff73c3 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:03.135242 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:03.169586 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 312/605] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:14:03.170835 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929da49-4311-4cf7-83dd-421ec4b787ed None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:03.210510 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99415bad-810d-4a50-a001-daa45bff73c3 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:03.251999 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fb5b3e-dbd2-46a2-9a0e-545318589a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:03.252652 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb5b3e-dbd2-46a2-9a0e-545318589a04 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:03.252922 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb5b3e-dbd2-46a2-9a0e-545318589a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:03.253186 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb5b3e-dbd2-46a2-9a0e-545318589a04 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:03.263938 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fb5b3e-dbd2-46a2-9a0e-545318589a04 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:03.265670 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fb5b3e-dbd2-46a2-9a0e-545318589a04 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:03.281663 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-58fb5b3e-dbd2-46a2-9a0e-545318589a04 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:03.281663 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:03.281663 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:03.281663 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:03.281663 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:03.281663 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:03.281663 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:03.281663 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:03.281663 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:03.281663 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:03.281663 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:03.281663 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:03.281663 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 10:14:03.281663 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 10:14:03.281663 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 10:14:03.282601 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 10:14:03.282601 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:03.282601 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:03.282601 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:03.282601 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:03.282601 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:03.282601 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:03.282601 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:03.282601 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:03.282601 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:03.282601 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:03.282601 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:03.282601 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:03.282601 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:03.282601 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:03.282601 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:03.282601 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:03.283550 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 10:14:03.283550 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 10:14:03.283550 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 10:14:03.283550 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 10:14:03.283550 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 10:14:03.283550 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 10:14:03.283550 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:03.283550 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:03.286444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 294/606] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:14:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:03.287586 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fb5b3e-dbd2-46a2-9a0e-545318589a04 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:03.296097 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929da49-4311-4cf7-83dd-421ec4b787ed None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:03.332594 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e5a0e-63a1-407a-b76a-6290ce2798b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:03.333506 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e5a0e-63a1-407a-b76a-6290ce2798b0 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:03.333877 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e5a0e-63a1-407a-b76a-6290ce2798b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:03.334965 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e5a0e-63a1-407a-b76a-6290ce2798b0 None admin] PATH_INFO: `/v3/projects/36b8b505012d4d0bac54845b6799e743/users/f70369fd5b334d888de0c091d8885eb3/roles/3a2b4319820a488198dabee34420cdc7` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:03.343827 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:03.356035 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653e5a0e-63a1-407a-b76a-6290ce2798b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=36b8b505012d4d0bac54845b6799e743, user_id=f70369fd5b334d888de0c091d8885eb3, role_id=3a2b4319820a488198dabee34420cdc7)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:03.358248 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653e5a0e-63a1-407a-b76a-6290ce2798b0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:03.379905 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 313/607] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 10:14:03 2022] PUT /identity/v3/projects/36b8b505012d4d0bac54845b6799e743/users/f70369fd5b334d888de0c091d8885eb3/roles/3a2b4319820a488198dabee34420cdc7 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:14:03.381007 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e5a0e-63a1-407a-b76a-6290ce2798b0 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:03.431910 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:03.445448 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fb5b3e-dbd2-46a2-9a0e-545318589a04 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:03.503730 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e5a0e-63a1-407a-b76a-6290ce2798b0 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:03.535808 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a8d10d-19b1-4e8f-b98c-3b8956667dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:03.536696 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a8d10d-19b1-4e8f-b98c-3b8956667dda None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:03.537009 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a8d10d-19b1-4e8f-b98c-3b8956667dda None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:03.537506 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a8d10d-19b1-4e8f-b98c-3b8956667dda None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:03.546136 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51afce8c-434e-4918-89ef-f834e4eb0751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:03.547130 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51afce8c-434e-4918-89ef-f834e4eb0751 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:03.547998 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51afce8c-434e-4918-89ef-f834e4eb0751 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:03.548410 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51afce8c-434e-4918-89ef-f834e4eb0751 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:03.549317 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51afce8c-434e-4918-89ef-f834e4eb0751 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:03.549770 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51afce8c-434e-4918-89ef-f834e4eb0751 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:03.551615 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51afce8c-434e-4918-89ef-f834e4eb0751 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:03.564653 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a8d10d-19b1-4e8f-b98c-3b8956667dda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:14:03.566949 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 314/608] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 10:14:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:03.575778 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-07a8d10d-19b1-4e8f-b98c-3b8956667dda None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 10:14:03.575778 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:03.575778 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:03.575778 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:03.575778 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:03.575778 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:03.575778 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:03.575778 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:03.575778 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:03.575778 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:03.575778 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:03.575778 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:03.575778 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:14:03.575778 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:14:03.575778 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:14:03.575778 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:14:03.577065 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:14:03.577065 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:14:03.577065 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:14:03.577065 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:14:03.577065 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:03.577065 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:03.577065 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:03.577065 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:03.577065 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:03.577065 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:03.577065 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:03.577065 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:03.577065 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:03.577065 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:03.577065 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:03.577065 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:03.578106 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:03.578106 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:03.578106 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:03.578106 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:03.578106 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:14:03.578106 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:14:03.578106 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:14:03.578106 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:14:03.578106 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:14:03.578106 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:14:03.578106 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 10:14:03.578106 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:03.578106 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 295/609] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 10:14:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:03.579352 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51afce8c-434e-4918-89ef-f834e4eb0751 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:03.583018 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a8d10d-19b1-4e8f-b98c-3b8956667dda None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:03.666704 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:03.723154 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a8d10d-19b1-4e8f-b98c-3b8956667dda None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:03.726276 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51afce8c-434e-4918-89ef-f834e4eb0751 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:03.760820 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a5f339-bd88-479a-a7c3-c1e3491f658d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:03.761982 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a5f339-bd88-479a-a7c3-c1e3491f658d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:03.762434 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a5f339-bd88-479a-a7c3-c1e3491f658d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:03.763086 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a5f339-bd88-479a-a7c3-c1e3491f658d None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:03.763504 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db43992-fedf-4252-b5b0-c3cc0a73acfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:03.764230 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a5f339-bd88-479a-a7c3-c1e3491f658d None admin] RBAC: Adding query filter params (name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:03.764413 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db43992-fedf-4252-b5b0-c3cc0a73acfd None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:03.764814 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db43992-fedf-4252-b5b0-c3cc0a73acfd None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:03.764957 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a5f339-bd88-479a-a7c3-c1e3491f658d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:03.765261 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db43992-fedf-4252-b5b0-c3cc0a73acfd None admin] PATH_INFO: `/v3/projects/36b8b505012d4d0bac54845b6799e743/users/a7deb94069a449269cb03406232174d3/roles/3a2b4319820a488198dabee34420cdc7` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:03.766961 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a5f339-bd88-479a-a7c3-c1e3491f658d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:03.780100 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 296/610] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 5 10:14:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:03.781027 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db43992-fedf-4252-b5b0-c3cc0a73acfd None admin] RBAC: Authorizing `identity:create_grant(project_id=36b8b505012d4d0bac54845b6799e743, user_id=a7deb94069a449269cb03406232174d3, role_id=3a2b4319820a488198dabee34420cdc7)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:03.782544 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db43992-fedf-4252-b5b0-c3cc0a73acfd None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:03.785316 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a5f339-bd88-479a-a7c3-c1e3491f658d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:03.805437 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:03.821133 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 315/611] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 10:14:03 2022] PUT /identity/v3/projects/36b8b505012d4d0bac54845b6799e743/users/a7deb94069a449269cb03406232174d3/roles/3a2b4319820a488198dabee34420cdc7 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:14:03.919433 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a5f339-bd88-479a-a7c3-c1e3491f658d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:03.973315 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e543ac-bf73-4e74-81e7-ca6da944a284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:03.974332 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e543ac-bf73-4e74-81e7-ca6da944a284 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:03.974809 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e543ac-bf73-4e74-81e7-ca6da944a284 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:03.975268 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e543ac-bf73-4e74-81e7-ca6da944a284 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:03.991380 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e543ac-bf73-4e74-81e7-ca6da944a284 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:03.993131 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e543ac-bf73-4e74-81e7-ca6da944a284 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:04.011644 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-61e543ac-bf73-4e74-81e7-ca6da944a284 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:04.011644 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:04.011644 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:04.011644 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:04.011644 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:04.011644 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:04.011644 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:04.011644 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:04.011644 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:04.011644 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:04.011644 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:04.011644 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:04.011644 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:04.011644 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:04.011644 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:04.012731 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:04.012731 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:04.012731 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:04.012731 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:04.012731 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:04.012731 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:04.012731 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:04.012731 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:04.012731 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:04.012731 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:04.012731 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:04.012731 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:04.012731 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:04.012731 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:04.012731 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:04.012731 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:04.012731 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:04.013633 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:04.013633 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:04.013633 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:04.013633 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:04.015343 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 297/612] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:14:04.024092 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db43992-fedf-4252-b5b0-c3cc0a73acfd None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:04.094228 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65801bc2-65c8-4b3f-a9ad-f9362e4c6474 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:04.094432 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65801bc2-65c8-4b3f-a9ad-f9362e4c6474 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:04.094696 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65801bc2-65c8-4b3f-a9ad-f9362e4c6474 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:04.096437 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 298/613] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:04.102986 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800aa76-7859-4d90-9388-235a5794c301 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:04.103277 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800aa76-7859-4d90-9388-235a5794c301 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:04.103555 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800aa76-7859-4d90-9388-235a5794c301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:04.125472 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-4800aa76-7859-4d90-9388-235a5794c301 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:04.126635 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4800aa76-7859-4d90-9388-235a5794c301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:04.187880 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:04.193627 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db43992-fedf-4252-b5b0-c3cc0a73acfd None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:04.222722 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:04.235225 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 299/614] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 10:14:04.240865 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d05c857-1ff8-4855-9098-1e3dfc4cfefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:04.241706 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05c857-1ff8-4855-9098-1e3dfc4cfefb None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:04.242122 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05c857-1ff8-4855-9098-1e3dfc4cfefb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:04.242553 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05c857-1ff8-4855-9098-1e3dfc4cfefb None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:04.243426 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d05c857-1ff8-4855-9098-1e3dfc4cfefb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:04.243864 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d05c857-1ff8-4855-9098-1e3dfc4cfefb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:04.245376 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d05c857-1ff8-4855-9098-1e3dfc4cfefb None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:04.256304 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4800aa76-7859-4d90-9388-235a5794c301 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:04.258099 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 316/615] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:04.269719 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d05c857-1ff8-4855-9098-1e3dfc4cfefb None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:04.378163 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4800aa76-7859-4d90-9388-235a5794c301 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:04.415636 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac051d5-94ed-4c84-96e2-55335900e0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:04.416315 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac051d5-94ed-4c84-96e2-55335900e0bc None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:04.416583 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac051d5-94ed-4c84-96e2-55335900e0bc None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:04.416852 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac051d5-94ed-4c84-96e2-55335900e0bc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:04.429244 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac051d5-94ed-4c84-96e2-55335900e0bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:14:04.434377 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-3ac051d5-94ed-4c84-96e2-55335900e0bc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 10:14:04.434377 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:04.434377 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:04.434377 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:04.434377 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:04.434377 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:04.434377 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:04.434377 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:04.434377 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:04.434377 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:04.434377 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:04.434377 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:04.434377 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:14:04.434377 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:14:04.434377 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:14:04.434377 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:14:04.435339 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:14:04.435339 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:14:04.435339 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:14:04.435339 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:14:04.435339 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:04.435339 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:04.435339 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:04.435339 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:04.435339 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:04.435339 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:04.435339 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:04.435339 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:04.435339 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:04.435339 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:04.435339 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:04.435339 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:04.436161 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:04.436161 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:04.436161 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:04.436161 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:04.436161 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:14:04.436161 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:14:04.436161 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:14:04.436161 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:14:04.436161 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:14:04.436161 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:14:04.436161 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 10:14:04.436161 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:04.436781 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 300/616] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 10:14:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:04.443971 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac051d5-94ed-4c84-96e2-55335900e0bc None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:04.497309 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d05c857-1ff8-4855-9098-1e3dfc4cfefb None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:04.520184 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:04.564494 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dba865-707a-4023-bdc4-09afb52a422f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:04.565238 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dba865-707a-4023-bdc4-09afb52a422f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:04.565445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dba865-707a-4023-bdc4-09afb52a422f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:04.565686 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dba865-707a-4023-bdc4-09afb52a422f None admin] PATH_INFO: `/v3/users/glance` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:04.569810 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac051d5-94ed-4c84-96e2-55335900e0bc None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:04.580160 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dba865-707a-4023-bdc4-09afb52a422f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:04.581625 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dba865-707a-4023-bdc4-09afb52a422f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:04.601618 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-29dba865-707a-4023-bdc4-09afb52a422f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 05 10:14:04.601618 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:04.601618 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:04.601618 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:04.601618 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:04.601618 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:04.601618 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:04.601618 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:04.601618 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:04.601618 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:04.601618 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:04.601618 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:04.601618 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 10:14:04.601618 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 10:14:04.601618 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 10:14:04.601618 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 10:14:04.602608 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:04.602608 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:04.602608 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 10:14:04.602608 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:04.602608 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 10:14:04.602608 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:04.602608 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:04.602608 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:04.602608 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:04.602608 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:04.602608 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:04.602608 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:04.602608 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:04.602608 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:04.602608 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:04.602608 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:04.602608 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:04.602608 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:04.603581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:04.603581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:04.603581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 10:14:04.603581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 10:14:04.603581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 10:14:04.603581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 10:14:04.603581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 10:14:04.603581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 10:14:04.603581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 05 10:14:04.603581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:04.603581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 317/617] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:14:04 2022] GET /identity/v3/users/glance => generated 84 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:04.609683 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dba865-707a-4023-bdc4-09afb52a422f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:04.615823 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d57f53d-30b7-4af3-864a-ed16bf07fa87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:04.616727 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d57f53d-30b7-4af3-864a-ed16bf07fa87 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:04.618245 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d57f53d-30b7-4af3-864a-ed16bf07fa87 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:04.618931 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d57f53d-30b7-4af3-864a-ed16bf07fa87 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:04.619852 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d57f53d-30b7-4af3-864a-ed16bf07fa87 None admin] RBAC: Adding query filter params (name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:04.623105 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d57f53d-30b7-4af3-864a-ed16bf07fa87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:04.627544 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d57f53d-30b7-4af3-864a-ed16bf07fa87 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:04.643243 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 301/618] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 5 10:14:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:14:04.651323 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:04.652198 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d57f53d-30b7-4af3-864a-ed16bf07fa87 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:04.802775 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dba865-707a-4023-bdc4-09afb52a422f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:04.842504 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d57f53d-30b7-4af3-864a-ed16bf07fa87 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:04.857755 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dbf571-f7f8-447f-9330-608cf245b112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:04.858427 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dbf571-f7f8-447f-9330-608cf245b112 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:04.858707 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dbf571-f7f8-447f-9330-608cf245b112 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:04.859005 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dbf571-f7f8-447f-9330-608cf245b112 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:04.859789 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dbf571-f7f8-447f-9330-608cf245b112 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:04.860093 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dbf571-f7f8-447f-9330-608cf245b112 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:04.861543 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dbf571-f7f8-447f-9330-608cf245b112 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:04.884527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:04.901718 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 318/619] 10.222.0.40 () {56 vars in 1137 bytes} [Fri Aug 5 10:14:04 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:04.907691 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dbf571-f7f8-447f-9330-608cf245b112 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:04.914597 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c55c3-613a-4a78-aa26-848222a881d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:04.915276 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c55c3-613a-4a78-aa26-848222a881d1 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:04.915553 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c55c3-613a-4a78-aa26-848222a881d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:04.915824 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c55c3-613a-4a78-aa26-848222a881d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:04.932491 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951c55c3-613a-4a78-aa26-848222a881d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:04.934886 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951c55c3-613a-4a78-aa26-848222a881d1 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:04.950804 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-951c55c3-613a-4a78-aa26-848222a881d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:04.950804 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:04.950804 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:04.950804 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:04.950804 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:04.950804 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:04.950804 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:04.950804 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:04.950804 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:04.950804 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:04.950804 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:04.950804 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:04.950804 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:04.950804 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:04.950804 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:04.951900 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:04.951900 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:04.951900 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:04.951900 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:04.951900 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:04.951900 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:04.951900 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:04.951900 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:04.951900 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:04.951900 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:04.951900 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:04.951900 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:04.951900 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:04.951900 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:04.951900 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:04.951900 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:04.951900 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:04.952785 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:04.952785 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:04.952785 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:04.952785 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:04.955051 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 302/620] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:14:04.964951 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c55c3-613a-4a78-aa26-848222a881d1 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:04.995759 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:05.020268 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dbf571-f7f8-447f-9330-608cf245b112 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:05.056039 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19192545-f767-4891-94b6-b8ca80715bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:05.056761 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19192545-f767-4891-94b6-b8ca80715bca None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:05.057075 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19192545-f767-4891-94b6-b8ca80715bca None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:05.057409 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19192545-f767-4891-94b6-b8ca80715bca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:05.085759 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c55c3-613a-4a78-aa26-848222a881d1 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:05.121156 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b7313-effd-4b5f-a514-cbe3ff0f6c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:05.121893 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b7313-effd-4b5f-a514-cbe3ff0f6c94 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:05.122174 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b7313-effd-4b5f-a514-cbe3ff0f6c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:05.122446 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b7313-effd-4b5f-a514-cbe3ff0f6c94 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:05.123181 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4b7313-effd-4b5f-a514-cbe3ff0f6c94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:05.123486 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4b7313-effd-4b5f-a514-cbe3ff0f6c94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:05.125133 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4b7313-effd-4b5f-a514-cbe3ff0f6c94 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:05.127090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19192545-f767-4891-94b6-b8ca80715bca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:05.128634 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19192545-f767-4891-94b6-b8ca80715bca None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:05.137892 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 303/621] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:05.139419 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0738afb-56e1-4004-af4b-66c248a209c3 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:05.139713 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0738afb-56e1-4004-af4b-66c248a209c3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:05.139991 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0738afb-56e1-4004-af4b-66c248a209c3 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:05.145416 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 304/622] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:05.146230 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0738afb-56e1-4004-af4b-66c248a209c3 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:05.180075 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:05.224742 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:05.254178 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 319/623] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:14:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 10:14:05.254178 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef25e01-10c8-4fcb-8423-2e13eb555271 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:05.254178 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef25e01-10c8-4fcb-8423-2e13eb555271 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:05.254178 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef25e01-10c8-4fcb-8423-2e13eb555271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:05.262818 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0738afb-56e1-4004-af4b-66c248a209c3 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:05.270252 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-4ef25e01-10c8-4fcb-8423-2e13eb555271 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:05.271562 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4ef25e01-10c8-4fcb-8423-2e13eb555271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:05.301334 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9361dd-5977-4fdb-93a5-4192060edc52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:05.302353 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9361dd-5977-4fdb-93a5-4192060edc52 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:05.302874 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9361dd-5977-4fdb-93a5-4192060edc52 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:05.303310 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9361dd-5977-4fdb-93a5-4192060edc52 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:05.311469 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9361dd-5977-4fdb-93a5-4192060edc52 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:05.313199 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9361dd-5977-4fdb-93a5-4192060edc52 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:05.326717 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-fc9361dd-5977-4fdb-93a5-4192060edc52 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 05 10:14:05.326717 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:05.326717 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:05.326717 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:05.326717 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:05.326717 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:05.326717 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:05.326717 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:05.326717 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:05.326717 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:05.326717 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:05.326717 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:05.326717 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 10:14:05.326717 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 10:14:05.326717 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 10:14:05.326717 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 10:14:05.327960 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:05.327960 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:05.327960 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 10:14:05.327960 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:05.327960 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 10:14:05.327960 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:05.327960 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:05.327960 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:05.327960 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:05.327960 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:05.327960 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:05.327960 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:05.327960 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:05.327960 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:05.327960 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:05.327960 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:05.327960 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:05.327960 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:05.329032 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:05.329032 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:05.329032 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 10:14:05.329032 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 10:14:05.329032 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 10:14:05.329032 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 10:14:05.329032 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 10:14:05.329032 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 10:14:05.329032 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 05 10:14:05.329032 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:05.331259 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 305/624] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 10:14:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:05.332304 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9361dd-5977-4fdb-93a5-4192060edc52 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:05.347430 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 320/625] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:14:05.348602 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef25e01-10c8-4fcb-8423-2e13eb555271 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:05.460741 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:05.462980 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef25e01-10c8-4fcb-8423-2e13eb555271 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:05.471392 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:05.471952 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9361dd-5977-4fdb-93a5-4192060edc52 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:05.499486 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ababbc60-fac8-4c76-b5e7-b809bfcb1af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:05.500283 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababbc60-fac8-4c76-b5e7-b809bfcb1af6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:05.500674 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababbc60-fac8-4c76-b5e7-b809bfcb1af6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:05.501052 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababbc60-fac8-4c76-b5e7-b809bfcb1af6 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:05.501959 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ababbc60-fac8-4c76-b5e7-b809bfcb1af6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:05.502375 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ababbc60-fac8-4c76-b5e7-b809bfcb1af6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:05.503946 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ababbc60-fac8-4c76-b5e7-b809bfcb1af6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:05.508330 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2260a6-4f61-44c2-b5fd-bd6a8710bbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:05.509021 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2260a6-4f61-44c2-b5fd-bd6a8710bbf2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:05.509300 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2260a6-4f61-44c2-b5fd-bd6a8710bbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:05.509569 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2260a6-4f61-44c2-b5fd-bd6a8710bbf2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:05.520997 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2260a6-4f61-44c2-b5fd-bd6a8710bbf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:05.522444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2260a6-4f61-44c2-b5fd-bd6a8710bbf2 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:05.523739 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 321/626] 10.222.0.40 () {56 vars in 1135 bytes} [Fri Aug 5 10:14:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:05.524497 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ababbc60-fac8-4c76-b5e7-b809bfcb1af6 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:05.542600 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-6c2260a6-4f61-44c2-b5fd-bd6a8710bbf2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:05.542600 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:05.542600 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:05.542600 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:05.542600 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:05.542600 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:05.542600 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:05.542600 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:05.542600 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:05.542600 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:05.542600 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:05.542600 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:05.542600 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:05.542600 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:05.542600 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:05.544076 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:05.544076 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:05.544076 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:05.544076 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:05.544076 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:05.544076 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:05.544076 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:05.544076 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:05.544076 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:05.544076 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:05.544076 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:05.544076 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:05.544076 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:05.544076 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:05.544076 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:05.544076 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:05.544076 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:05.545175 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:05.545175 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:05.545175 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:05.545175 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:05.545175 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 306/627] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:05.550012 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2260a6-4f61-44c2-b5fd-bd6a8710bbf2 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:05.627510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ababbc60-fac8-4c76-b5e7-b809bfcb1af6 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:05.650858 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2260a6-4f61-44c2-b5fd-bd6a8710bbf2 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:05.667484 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213abf63-4c34-481a-bb99-db22ab0dfdcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:05.669107 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213abf63-4c34-481a-bb99-db22ab0dfdcc None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:05.669529 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213abf63-4c34-481a-bb99-db22ab0dfdcc None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:05.669905 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213abf63-4c34-481a-bb99-db22ab0dfdcc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:05.681906 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213abf63-4c34-481a-bb99-db22ab0dfdcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:14:05.687942 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87e775-cd77-4719-aab0-4530c4f75844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:05.688601 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87e775-cd77-4719-aab0-4530c4f75844 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:05.688878 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87e775-cd77-4719-aab0-4530c4f75844 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:05.689148 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87e775-cd77-4719-aab0-4530c4f75844 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:05.691982 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-213abf63-4c34-481a-bb99-db22ab0dfdcc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 10:14:05.691982 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:05.691982 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:05.691982 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:05.691982 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:05.691982 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:05.691982 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:05.691982 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:05.691982 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:05.691982 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:05.691982 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:05.691982 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:05.691982 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:14:05.691982 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:14:05.691982 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:14:05.691982 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:14:05.693105 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:14:05.693105 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:14:05.693105 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:14:05.693105 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:14:05.693105 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:05.693105 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:05.693105 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:05.693105 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:05.693105 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:05.693105 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:05.693105 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:05.693105 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:05.693105 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:05.693105 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:05.693105 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:05.693105 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:05.694200 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:05.694200 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:05.694200 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:05.694200 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:05.694200 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:14:05.694200 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:14:05.694200 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:14:05.694200 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:14:05.694200 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:14:05.694200 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:14:05.694200 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 10:14:05.694200 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:05.696811 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 322/628] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 10:14:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:05.701583 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213abf63-4c34-481a-bb99-db22ab0dfdcc None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:05.706048 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a87e775-cd77-4719-aab0-4530c4f75844 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:05.707710 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a87e775-cd77-4719-aab0-4530c4f75844 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:05.710674 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:05.724890 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:05.726816 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-0a87e775-cd77-4719-aab0-4530c4f75844 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:05.726816 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:05.726816 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:05.726816 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:05.726816 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:05.726816 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:05.726816 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:05.726816 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:05.726816 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:05.726816 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:05.726816 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:05.726816 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:05.726816 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:05.726816 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:05.726816 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:05.728228 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:05.728228 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:05.728228 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:05.728228 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:05.728228 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:05.728228 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:05.728228 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:05.728228 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:05.728228 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:05.728228 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:05.728228 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:05.728228 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:05.728228 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:05.728228 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:05.728228 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:05.728228 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:05.728228 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:05.729367 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:05.729367 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:05.729367 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:05.729367 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:05.736569 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 307/629] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:05.738648 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87e775-cd77-4719-aab0-4530c4f75844 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:05.803009 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213abf63-4c34-481a-bb99-db22ab0dfdcc None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:05.843649 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25651c6-fc11-41a5-820f-b43793da9f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:05.844399 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25651c6-fc11-41a5-820f-b43793da9f40 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:05.844681 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25651c6-fc11-41a5-820f-b43793da9f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:05.844947 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25651c6-fc11-41a5-820f-b43793da9f40 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:05.845629 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25651c6-fc11-41a5-820f-b43793da9f40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:05.845867 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87e775-cd77-4719-aab0-4530c4f75844 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:05.845967 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25651c6-fc11-41a5-820f-b43793da9f40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:05.847317 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25651c6-fc11-41a5-820f-b43793da9f40 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:05.861737 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 323/630] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:05.863208 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25651c6-fc11-41a5-820f-b43793da9f40 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:05.881913 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2529e4-61a5-4349-bf73-1a604ce69000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:05.882915 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2529e4-61a5-4349-bf73-1a604ce69000 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:05.883216 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2529e4-61a5-4349-bf73-1a604ce69000 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:05.883684 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2529e4-61a5-4349-bf73-1a604ce69000 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:05.884350 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2529e4-61a5-4349-bf73-1a604ce69000 None admin] RBAC: Adding query filter params (name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:05.884648 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2529e4-61a5-4349-bf73-1a604ce69000 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:05.886019 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2529e4-61a5-4349-bf73-1a604ce69000 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:05.899347 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 308/631] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 5 10:14:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:05.899347 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1d52b-ad81-470d-8469-b95c7175a1ab None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:05.899347 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1d52b-ad81-470d-8469-b95c7175a1ab None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:05.899347 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1d52b-ad81-470d-8469-b95c7175a1ab None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:05.899347 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 309/632] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:05.899347 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c1d52b-ad81-470d-8469-b95c7175a1ab None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:05.964139 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:05.965998 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:05.981751 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25651c6-fc11-41a5-820f-b43793da9f40 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:06.014147 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c1d52b-ad81-470d-8469-b95c7175a1ab None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:06.016402 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd41fcf-1f05-405f-8e87-53a28808bff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:06.017034 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd41fcf-1f05-405f-8e87-53a28808bff7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:06.017305 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd41fcf-1f05-405f-8e87-53a28808bff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:06.017573 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd41fcf-1f05-405f-8e87-53a28808bff7 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:06.018266 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd41fcf-1f05-405f-8e87-53a28808bff7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:06.018564 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd41fcf-1f05-405f-8e87-53a28808bff7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:06.019981 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd41fcf-1f05-405f-8e87-53a28808bff7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:06.033101 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 324/633] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:06.033719 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd41fcf-1f05-405f-8e87-53a28808bff7 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:06.058433 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d83dbb-f10b-4d8b-8201-8b1cade3c299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:06.059109 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d83dbb-f10b-4d8b-8201-8b1cade3c299 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:06.059567 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d83dbb-f10b-4d8b-8201-8b1cade3c299 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:06.059816 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d83dbb-f10b-4d8b-8201-8b1cade3c299 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:06.067457 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d83dbb-f10b-4d8b-8201-8b1cade3c299 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:06.068888 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d83dbb-f10b-4d8b-8201-8b1cade3c299 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:06.080245 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-65d83dbb-f10b-4d8b-8201-8b1cade3c299 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:06.080245 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:06.080245 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:06.080245 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:06.080245 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:06.080245 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:06.080245 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:06.080245 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:06.080245 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:06.080245 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:06.080245 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:06.080245 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:06.080245 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 10:14:06.080245 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 10:14:06.080245 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 10:14:06.081206 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 10:14:06.081206 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:06.081206 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:06.081206 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:06.081206 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:06.081206 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:06.081206 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:06.081206 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:06.081206 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:06.081206 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:06.081206 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:06.081206 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:06.081206 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:06.081206 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:06.081206 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:06.081206 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:06.081206 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:06.082098 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 10:14:06.082098 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 10:14:06.082098 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 10:14:06.082098 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 10:14:06.082098 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 10:14:06.082098 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 10:14:06.082098 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:06.082098 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:06.082098 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 310/634] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:14:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:06.082594 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6859afe-ccd0-434a-bff2-2c9c2e0eee51 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:06.084274 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6859afe-ccd0-434a-bff2-2c9c2e0eee51 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:06.084274 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6859afe-ccd0-434a-bff2-2c9c2e0eee51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:06.100565 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-e6859afe-ccd0-434a-bff2-2c9c2e0eee51 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:06.101722 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e6859afe-ccd0-434a-bff2-2c9c2e0eee51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:06.145110 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd41fcf-1f05-405f-8e87-53a28808bff7 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:06.175854 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 311/635] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:14:06.176522 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6859afe-ccd0-434a-bff2-2c9c2e0eee51 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:06.196040 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6962e65c-329f-4a23-9152-81465610a154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:06.197029 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962e65c-329f-4a23-9152-81465610a154 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:06.198061 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962e65c-329f-4a23-9152-81465610a154 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:06.198342 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962e65c-329f-4a23-9152-81465610a154 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:06.213505 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6962e65c-329f-4a23-9152-81465610a154 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:06.215026 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6962e65c-329f-4a23-9152-81465610a154 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:06.218381 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:06.226923 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:06.234639 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-6962e65c-329f-4a23-9152-81465610a154 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:06.234639 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:06.234639 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:06.234639 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:06.234639 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:06.234639 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:06.234639 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:06.234639 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:06.234639 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:06.234639 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:06.234639 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:06.234639 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:06.234639 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:06.234639 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:06.234639 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:06.235589 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:06.235589 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:06.235589 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:06.235589 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:06.235589 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:06.235589 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:06.235589 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:06.235589 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:06.235589 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:06.235589 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:06.235589 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:06.235589 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:06.235589 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:06.235589 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:06.235589 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:06.235589 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:06.235589 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:06.236479 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:06.236479 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:06.236479 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:06.236479 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:06.236479 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 325/636] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:06.236749 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6962e65c-329f-4a23-9152-81465610a154 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:06.286115 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6859afe-ccd0-434a-bff2-2c9c2e0eee51 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:06.321387 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f2c2fe-c8a2-4cec-acb4-4a56269b878d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:06.322041 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2c2fe-c8a2-4cec-acb4-4a56269b878d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:06.322308 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2c2fe-c8a2-4cec-acb4-4a56269b878d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:06.322573 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2c2fe-c8a2-4cec-acb4-4a56269b878d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:06.329465 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f2c2fe-c8a2-4cec-acb4-4a56269b878d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:06.330945 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f2c2fe-c8a2-4cec-acb4-4a56269b878d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:06.339495 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6962e65c-329f-4a23-9152-81465610a154 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:06.342598 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-16f2c2fe-c8a2-4cec-acb4-4a56269b878d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:06.342598 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:06.342598 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:06.342598 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:06.342598 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:06.342598 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:06.342598 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:06.342598 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:06.342598 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:06.342598 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:06.342598 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:06.342598 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:06.342598 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 10:14:06.342598 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 10:14:06.342598 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 10:14:06.343621 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 10:14:06.343621 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:06.343621 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:06.343621 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:06.343621 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:06.343621 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:06.343621 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:06.343621 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:06.343621 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:06.343621 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:06.343621 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:06.343621 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:06.343621 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:06.343621 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:06.343621 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:06.343621 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:06.343621 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:06.349574 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 10:14:06.349574 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 10:14:06.349574 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 10:14:06.349574 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 10:14:06.349574 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 10:14:06.349574 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 10:14:06.349574 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:06.349574 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:06.349574 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 312/637] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:14:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:06.349574 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f2c2fe-c8a2-4cec-acb4-4a56269b878d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:06.376343 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce10c12a-61ef-492b-9ca6-a95b256657e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:06.376970 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10c12a-61ef-492b-9ca6-a95b256657e9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:06.377258 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10c12a-61ef-492b-9ca6-a95b256657e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:06.377526 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10c12a-61ef-492b-9ca6-a95b256657e9 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:06.378275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce10c12a-61ef-492b-9ca6-a95b256657e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:06.378584 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce10c12a-61ef-492b-9ca6-a95b256657e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:06.380062 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce10c12a-61ef-492b-9ca6-a95b256657e9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:06.392402 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 326/638] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 10:14:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:06.393226 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce10c12a-61ef-492b-9ca6-a95b256657e9 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:06.447902 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:06.453068 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f2c2fe-c8a2-4cec-acb4-4a56269b878d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:06.462180 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:06.486555 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daba6f0-ea6e-4194-8329-b1bbd6d6f0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:06.487156 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daba6f0-ea6e-4194-8329-b1bbd6d6f0c3 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:06.487431 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daba6f0-ea6e-4194-8329-b1bbd6d6f0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:06.487700 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daba6f0-ea6e-4194-8329-b1bbd6d6f0c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:06.492388 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce10c12a-61ef-492b-9ca6-a95b256657e9 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:06.498237 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daba6f0-ea6e-4194-8329-b1bbd6d6f0c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:14:06.503116 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-7daba6f0-ea6e-4194-8329-b1bbd6d6f0c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 10:14:06.503116 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:06.503116 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:06.503116 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:06.503116 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:06.503116 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:06.503116 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:06.503116 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:06.503116 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:06.503116 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:06.503116 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:06.503116 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:06.503116 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:14:06.503116 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:14:06.503116 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:14:06.503116 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:14:06.504274 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:14:06.504274 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:14:06.504274 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:14:06.504274 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:14:06.504274 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:06.504274 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:06.504274 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:06.504274 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:06.504274 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:06.504274 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:06.504274 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:06.504274 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:06.504274 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:06.504274 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:06.504274 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:06.504274 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:06.505332 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:06.505332 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:06.505332 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:06.505332 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:06.505332 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:14:06.505332 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:14:06.505332 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:14:06.505332 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:14:06.505332 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:14:06.505332 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:14:06.505332 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 10:14:06.505332 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:06.505332 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 313/639] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 10:14:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:06.506184 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de65ee-9f0c-4cbf-b1d2-85bd83022256 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:06.506184 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de65ee-9f0c-4cbf-b1d2-85bd83022256 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:06.506184 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de65ee-9f0c-4cbf-b1d2-85bd83022256 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:06.510988 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 314/640] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:06.510988 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10de65ee-9f0c-4cbf-b1d2-85bd83022256 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:06.532255 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cf34e1-08dc-4aea-9740-bb925a8353e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:06.532882 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf34e1-08dc-4aea-9740-bb925a8353e0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:06.533161 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf34e1-08dc-4aea-9740-bb925a8353e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:06.533434 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf34e1-08dc-4aea-9740-bb925a8353e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:06.534123 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cf34e1-08dc-4aea-9740-bb925a8353e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:06.534426 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cf34e1-08dc-4aea-9740-bb925a8353e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:06.535839 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cf34e1-08dc-4aea-9740-bb925a8353e0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:06.547981 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 327/641] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:06.548763 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cf34e1-08dc-4aea-9740-bb925a8353e0 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:06.610594 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10de65ee-9f0c-4cbf-b1d2-85bd83022256 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:06.644069 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e83c7d-0129-4f22-a524-e4bdc2b6d0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:06.644759 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e83c7d-0129-4f22-a524-e4bdc2b6d0db None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:06.645132 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e83c7d-0129-4f22-a524-e4bdc2b6d0db None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:06.645402 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e83c7d-0129-4f22-a524-e4bdc2b6d0db None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:06.646141 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e83c7d-0129-4f22-a524-e4bdc2b6d0db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:06.646443 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e83c7d-0129-4f22-a524-e4bdc2b6d0db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:06.647907 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e83c7d-0129-4f22-a524-e4bdc2b6d0db None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:06.648910 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cf34e1-08dc-4aea-9740-bb925a8353e0 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:06.659618 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 315/642] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 10:14:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:06.660397 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e83c7d-0129-4f22-a524-e4bdc2b6d0db None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:06.665474 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:06.682453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df068a41-dc2e-4760-b690-82d2996701a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:06.683097 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df068a41-dc2e-4760-b690-82d2996701a1 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:06.683374 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df068a41-dc2e-4760-b690-82d2996701a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:06.683643 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df068a41-dc2e-4760-b690-82d2996701a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:06.684721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df068a41-dc2e-4760-b690-82d2996701a1 None admin] RBAC: Adding query filter params (role.id=3a2b4319820a488198dabee34420cdc7, scope.project.id=36b8b505012d4d0bac54845b6799e743, user.id=ac0a8cf7f8b74e88a04444ec38547528) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:06.685042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df068a41-dc2e-4760-b690-82d2996701a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:06.686476 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df068a41-dc2e-4760-b690-82d2996701a1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:06.690195 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:06.692312 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 328/643] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:14:06 2022] GET /identity/v3/role_assignments?user.id=ac0a8cf7f8b74e88a04444ec38547528&scope.project.id=36b8b505012d4d0bac54845b6799e743&role.id=3a2b4319820a488198dabee34420cdc7 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:06.693464 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6993702-1a95-4b1c-ac80-9341c1277da2 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:06.693746 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6993702-1a95-4b1c-ac80-9341c1277da2 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:06.694021 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6993702-1a95-4b1c-ac80-9341c1277da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:06.714011 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a6993702-1a95-4b1c-ac80-9341c1277da2 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:06.715270 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a6993702-1a95-4b1c-ac80-9341c1277da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:06.773452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e83c7d-0129-4f22-a524-e4bdc2b6d0db None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:06.799060 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 329/644] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:14:06.800001 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6993702-1a95-4b1c-ac80-9341c1277da2 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:06.824476 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df199f39-9177-47de-86fa-235344d302e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:06.825176 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df199f39-9177-47de-86fa-235344d302e2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:06.825464 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df199f39-9177-47de-86fa-235344d302e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:06.825735 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df199f39-9177-47de-86fa-235344d302e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:06.826470 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df199f39-9177-47de-86fa-235344d302e2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:06.826844 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df199f39-9177-47de-86fa-235344d302e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:06.828288 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df199f39-9177-47de-86fa-235344d302e2 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:06.837259 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 316/645] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 5 10:14:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:06.838094 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df199f39-9177-47de-86fa-235344d302e2 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:06.924503 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6993702-1a95-4b1c-ac80-9341c1277da2 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:06.936662 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:06.949010 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:06.959900 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df199f39-9177-47de-86fa-235344d302e2 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:06.961594 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fb2901-99d8-470f-8451-a8f7b88f912b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:06.962327 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb2901-99d8-470f-8451-a8f7b88f912b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:06.962672 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb2901-99d8-470f-8451-a8f7b88f912b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:06.963044 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb2901-99d8-470f-8451-a8f7b88f912b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:06.971643 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fb2901-99d8-470f-8451-a8f7b88f912b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:06.973165 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fb2901-99d8-470f-8451-a8f7b88f912b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:06.985090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-71fb2901-99d8-470f-8451-a8f7b88f912b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 10:14:06.985090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:06.985090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:06.985090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:06.985090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:06.985090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:06.985090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:06.985090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:06.985090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:06.985090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:06.985090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:06.985090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:06.985090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 10:14:06.985090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 10:14:06.985090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 10:14:06.985090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 10:14:06.986044 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:06.986044 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:06.986044 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 10:14:06.986044 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:06.986044 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 10:14:06.986044 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:06.986044 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:06.986044 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:06.986044 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:06.986044 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:06.986044 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:06.986044 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:06.986044 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:06.986044 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:06.986044 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:06.986044 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:06.986044 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:06.986044 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:06.987148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:06.987148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:06.987148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 10:14:06.987148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 10:14:06.987148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 10:14:06.987148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 10:14:06.987148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 10:14:06.987148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 10:14:06.987148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 10:14:06.987148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:06.987148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 330/646] 10.222.0.40 () {56 vars in 1093 bytes} [Fri Aug 5 10:14:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:06.987148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fb2901-99d8-470f-8451-a8f7b88f912b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:07.001865 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39590286-6d6c-4406-a58d-236f2236e72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:07.003222 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39590286-6d6c-4406-a58d-236f2236e72a None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:07.003696 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39590286-6d6c-4406-a58d-236f2236e72a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:07.004138 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39590286-6d6c-4406-a58d-236f2236e72a None admin] PATH_INFO: `/v3/projects/36b8b505012d4d0bac54845b6799e743/users/c80cf0e429f04f229aea1aa9293a93b2/roles/3a2b4319820a488198dabee34420cdc7` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:07.020839 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39590286-6d6c-4406-a58d-236f2236e72a None admin] RBAC: Authorizing `identity:create_grant(project_id=36b8b505012d4d0bac54845b6799e743, user_id=c80cf0e429f04f229aea1aa9293a93b2, role_id=3a2b4319820a488198dabee34420cdc7)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:07.022566 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39590286-6d6c-4406-a58d-236f2236e72a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:07.038642 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 317/647] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 10:14:06 2022] PUT /identity/v3/projects/36b8b505012d4d0bac54845b6799e743/users/c80cf0e429f04f229aea1aa9293a93b2/roles/3a2b4319820a488198dabee34420cdc7 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:14:07.039849 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39590286-6d6c-4406-a58d-236f2236e72a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:07.118070 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fb2901-99d8-470f-8451-a8f7b88f912b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:07.171857 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e09c89c-827b-4c21-ab47-1827b7d94fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:07.173328 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e09c89c-827b-4c21-ab47-1827b7d94fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:07.173828 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e09c89c-827b-4c21-ab47-1827b7d94fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:07.174277 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e09c89c-827b-4c21-ab47-1827b7d94fb1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:07.187587 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e09c89c-827b-4c21-ab47-1827b7d94fb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:14:07.193671 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-4e09c89c-827b-4c21-ab47-1827b7d94fb1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 10:14:07.193671 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:07.193671 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:07.193671 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:07.193671 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:07.193671 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:07.193671 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:07.193671 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:07.193671 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:07.193671 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:07.193671 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:07.193671 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:07.193671 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:14:07.193671 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:14:07.193671 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:14:07.193671 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:14:07.194659 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:14:07.194659 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:14:07.194659 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:14:07.194659 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:14:07.194659 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:07.194659 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:07.194659 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:07.194659 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:07.194659 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:07.194659 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:07.194659 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:07.194659 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:07.194659 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:07.194659 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:07.194659 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:07.194659 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:07.195547 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:07.195547 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:07.195547 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:07.195547 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:07.195547 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:14:07.195547 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:14:07.195547 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:14:07.195547 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:14:07.195547 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:14:07.195547 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:14:07.195547 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 10:14:07.195547 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:07.196185 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39590286-6d6c-4406-a58d-236f2236e72a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:07.197768 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 331/648] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 10:14:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:07.198797 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e09c89c-827b-4c21-ab47-1827b7d94fb1 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:07.239974 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fef5bf-2db0-41f8-9bc7-25d8e668fe89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:07.240886 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fef5bf-2db0-41f8-9bc7-25d8e668fe89 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:07.241542 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fef5bf-2db0-41f8-9bc7-25d8e668fe89 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:07.241896 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fef5bf-2db0-41f8-9bc7-25d8e668fe89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:07.243615 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:07.254052 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:07.256718 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fef5bf-2db0-41f8-9bc7-25d8e668fe89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:07.258199 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fef5bf-2db0-41f8-9bc7-25d8e668fe89 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:07.280990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-23fef5bf-2db0-41f8-9bc7-25d8e668fe89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:07.280990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:07.280990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:07.280990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:07.280990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:07.280990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:07.280990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:07.280990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:07.280990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:07.280990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:07.280990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:07.280990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:07.280990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:07.280990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:07.280990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:07.281957 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:07.281957 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:07.281957 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:07.281957 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:07.281957 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:07.281957 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:07.281957 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:07.281957 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:07.281957 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:07.281957 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:07.281957 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:07.281957 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:07.281957 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:07.281957 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:07.281957 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:07.281957 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:07.281957 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:07.282864 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:07.282864 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:07.282864 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:07.282864 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:07.283523 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 318/649] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:07.287346 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fef5bf-2db0-41f8-9bc7-25d8e668fe89 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:07.339063 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e09c89c-827b-4c21-ab47-1827b7d94fb1 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:07.376078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c69888-bf57-4518-bc9b-89b9a353965e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:07.376716 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c69888-bf57-4518-bc9b-89b9a353965e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:07.377088 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c69888-bf57-4518-bc9b-89b9a353965e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:07.377368 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c69888-bf57-4518-bc9b-89b9a353965e None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:07.378167 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c69888-bf57-4518-bc9b-89b9a353965e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:07.378481 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c69888-bf57-4518-bc9b-89b9a353965e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:07.380166 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c69888-bf57-4518-bc9b-89b9a353965e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:07.391891 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fef5bf-2db0-41f8-9bc7-25d8e668fe89 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:07.406216 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 332/650] 10.222.0.40 () {56 vars in 1133 bytes} [Fri Aug 5 10:14:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:07.407202 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c69888-bf57-4518-bc9b-89b9a353965e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:07.427098 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c09f92-e4c9-4a24-b4a2-750601bf7772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:07.427703 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c09f92-e4c9-4a24-b4a2-750601bf7772 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:07.427979 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c09f92-e4c9-4a24-b4a2-750601bf7772 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:07.428249 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c09f92-e4c9-4a24-b4a2-750601bf7772 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:07.428964 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c09f92-e4c9-4a24-b4a2-750601bf7772 None admin] RBAC: Adding query filter params (name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:07.429264 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c09f92-e4c9-4a24-b4a2-750601bf7772 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:07.430628 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c09f92-e4c9-4a24-b4a2-750601bf7772 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:07.439671 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 319/651] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 5 10:14:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:07.440525 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c09f92-e4c9-4a24-b4a2-750601bf7772 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:07.497531 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:07.531796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:07.545886 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c09f92-e4c9-4a24-b4a2-750601bf7772 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:07.548747 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c69888-bf57-4518-bc9b-89b9a353965e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:07.581495 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d36ce-5f3d-40ce-ba42-c1f55d54299d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:07.582163 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d36ce-5f3d-40ce-ba42-c1f55d54299d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:07.582812 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d36ce-5f3d-40ce-ba42-c1f55d54299d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:07.582812 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d36ce-5f3d-40ce-ba42-c1f55d54299d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:07.599286 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad94f7d-59d2-4fa4-86bd-15fe0cd5149c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:07.600338 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad94f7d-59d2-4fa4-86bd-15fe0cd5149c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:07.600849 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad94f7d-59d2-4fa4-86bd-15fe0cd5149c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:07.601573 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad94f7d-59d2-4fa4-86bd-15fe0cd5149c None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:07.603309 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad94f7d-59d2-4fa4-86bd-15fe0cd5149c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:07.604019 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad94f7d-59d2-4fa4-86bd-15fe0cd5149c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:07.606997 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad94f7d-59d2-4fa4-86bd-15fe0cd5149c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:07.627256 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 333/652] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:07.628407 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad94f7d-59d2-4fa4-86bd-15fe0cd5149c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:07.652167 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850d36ce-5f3d-40ce-ba42-c1f55d54299d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:07.653883 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850d36ce-5f3d-40ce-ba42-c1f55d54299d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:07.762383 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad94f7d-59d2-4fa4-86bd-15fe0cd5149c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:07.787263 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:07.789314 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 320/653] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:14:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 10:14:07.790484 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d36ce-5f3d-40ce-ba42-c1f55d54299d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:07.808805 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ff1ae-f013-4626-bb48-5d9edad4643a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:07.809979 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ff1ae-f013-4626-bb48-5d9edad4643a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:07.810453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ff1ae-f013-4626-bb48-5d9edad4643a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:07.810922 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ff1ae-f013-4626-bb48-5d9edad4643a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:07.825569 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643ff1ae-f013-4626-bb48-5d9edad4643a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:07.827091 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643ff1ae-f013-4626-bb48-5d9edad4643a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:07.842177 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:07.851393 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-643ff1ae-f013-4626-bb48-5d9edad4643a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:07.851393 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:07.851393 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:07.851393 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:07.851393 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:07.851393 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:07.851393 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:07.851393 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:07.851393 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:07.851393 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:07.851393 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:07.851393 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:07.851393 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:07.851393 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:07.851393 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:07.852341 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:07.852341 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:07.852341 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:07.852341 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:07.852341 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:07.852341 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:07.852341 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:07.852341 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:07.852341 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:07.852341 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:07.852341 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:07.852341 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:07.852341 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:07.852341 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:07.852341 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:07.852341 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:07.852341 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:07.853418 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:07.853418 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:07.853418 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:07.853418 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:07.860028 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 334/654] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:07.861204 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ff1ae-f013-4626-bb48-5d9edad4643a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:07.915481 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d36ce-5f3d-40ce-ba42-c1f55d54299d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:07.960740 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a99eba-24af-4809-aece-bfc6010d89d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:07.961405 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a99eba-24af-4809-aece-bfc6010d89d6 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:07.961680 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a99eba-24af-4809-aece-bfc6010d89d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:07.962381 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a99eba-24af-4809-aece-bfc6010d89d6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:07.971739 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a99eba-24af-4809-aece-bfc6010d89d6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:07.974008 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a99eba-24af-4809-aece-bfc6010d89d6 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:07.989005 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ff1ae-f013-4626-bb48-5d9edad4643a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:07.989354 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-76a99eba-24af-4809-aece-bfc6010d89d6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 10:14:07.989354 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:07.989354 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:07.989354 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:07.989354 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:07.989354 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:07.989354 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:07.989354 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:07.989354 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:07.989354 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:07.989354 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:07.989354 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:07.989354 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 10:14:07.989354 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 10:14:07.989354 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 10:14:07.989354 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 10:14:07.990315 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:07.990315 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:07.990315 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 10:14:07.990315 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:07.990315 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 10:14:07.990315 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:07.990315 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:07.990315 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:07.990315 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:07.990315 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:07.990315 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:07.990315 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:07.990315 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:07.990315 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:07.990315 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:07.990315 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:07.990315 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:07.990315 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:07.991300 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:07.991300 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:07.991300 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 10:14:07.991300 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 10:14:07.991300 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 10:14:07.991300 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 10:14:07.991300 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 10:14:07.991300 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 10:14:07.991300 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 10:14:07.991300 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:07.994978 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 321/655] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:14:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:07.996115 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a99eba-24af-4809-aece-bfc6010d89d6 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:08.023568 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:08.038577 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d686ea35-7bd6-446b-80aa-838a6ec6b6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:08.043215 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686ea35-7bd6-446b-80aa-838a6ec6b6da None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:08.044131 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686ea35-7bd6-446b-80aa-838a6ec6b6da None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:08.044689 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686ea35-7bd6-446b-80aa-838a6ec6b6da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:08.062544 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d686ea35-7bd6-446b-80aa-838a6ec6b6da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:08.064520 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d686ea35-7bd6-446b-80aa-838a6ec6b6da None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:08.087843 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-d686ea35-7bd6-446b-80aa-838a6ec6b6da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:08.087843 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:08.087843 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:08.087843 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:08.087843 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:08.087843 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:08.087843 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:08.087843 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:08.087843 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:08.087843 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:08.087843 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:08.087843 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:08.087843 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:08.087843 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:08.087843 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:08.089493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:08.089493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:08.089493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:08.089493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:08.089493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:08.089493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:08.089493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:08.089493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:08.089493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:08.089493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:08.089493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:08.089493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:08.089493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:08.089493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:08.089493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:08.089493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:08.089493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:08.091226 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:08.091226 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:08.091226 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:08.091226 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:08.098890 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 335/656] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:08.098890 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d686ea35-7bd6-446b-80aa-838a6ec6b6da None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:08.131470 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a99eba-24af-4809-aece-bfc6010d89d6 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:08.178242 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab650ebf-d4b2-42ae-83c1-f31ba4d1345b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:08.178242 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab650ebf-d4b2-42ae-83c1-f31ba4d1345b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:08.178242 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab650ebf-d4b2-42ae-83c1-f31ba4d1345b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:08.178242 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab650ebf-d4b2-42ae-83c1-f31ba4d1345b None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:08.178888 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab650ebf-d4b2-42ae-83c1-f31ba4d1345b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:08.178888 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab650ebf-d4b2-42ae-83c1-f31ba4d1345b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:08.178888 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab650ebf-d4b2-42ae-83c1-f31ba4d1345b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:08.184061 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:08.199030 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 322/657] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:08.199030 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab650ebf-d4b2-42ae-83c1-f31ba4d1345b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:08.262309 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d686ea35-7bd6-446b-80aa-838a6ec6b6da None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:08.284985 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:08.312290 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf614f2-0089-4d99-9f8e-b9964d98d751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:08.312504 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab650ebf-d4b2-42ae-83c1-f31ba4d1345b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:08.313422 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf614f2-0089-4d99-9f8e-b9964d98d751 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:08.314026 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf614f2-0089-4d99-9f8e-b9964d98d751 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:08.314544 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf614f2-0089-4d99-9f8e-b9964d98d751 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:08.320091 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf614f2-0089-4d99-9f8e-b9964d98d751 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:08.320767 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf614f2-0089-4d99-9f8e-b9964d98d751 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:08.323305 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf614f2-0089-4d99-9f8e-b9964d98d751 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:08.348107 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 336/658] 10.222.0.40 () {56 vars in 1137 bytes} [Fri Aug 5 10:14:08 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:08.349405 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf614f2-0089-4d99-9f8e-b9964d98d751 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:08.355220 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad899f13-24a7-429c-a1aa-e387fd9e42f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:08.355853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad899f13-24a7-429c-a1aa-e387fd9e42f2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:08.356180 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad899f13-24a7-429c-a1aa-e387fd9e42f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:08.356445 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad899f13-24a7-429c-a1aa-e387fd9e42f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:08.357165 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad899f13-24a7-429c-a1aa-e387fd9e42f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:08.358404 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad899f13-24a7-429c-a1aa-e387fd9e42f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:08.359815 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad899f13-24a7-429c-a1aa-e387fd9e42f2 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:08.374344 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 323/659] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:08.379649 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad899f13-24a7-429c-a1aa-e387fd9e42f2 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:08.489823 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf614f2-0089-4d99-9f8e-b9964d98d751 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:08.494865 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad899f13-24a7-429c-a1aa-e387fd9e42f2 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:08.502342 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:08.535273 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8339d9-db65-47c1-848b-a8c7c978863b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:08.536287 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a4954c-49b9-45b3-bc78-ac721bbec26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:08.536532 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8339d9-db65-47c1-848b-a8c7c978863b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:08.536943 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4954c-49b9-45b3-bc78-ac721bbec26a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:08.537220 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4954c-49b9-45b3-bc78-ac721bbec26a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:08.537429 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8339d9-db65-47c1-848b-a8c7c978863b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:08.537527 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4954c-49b9-45b3-bc78-ac721bbec26a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:08.537979 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8339d9-db65-47c1-848b-a8c7c978863b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:08.541508 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:08.551481 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a4954c-49b9-45b3-bc78-ac721bbec26a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:08.554168 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a4954c-49b9-45b3-bc78-ac721bbec26a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:08.569804 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-67a4954c-49b9-45b3-bc78-ac721bbec26a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 05 10:14:08.569804 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:08.569804 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:08.569804 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:08.569804 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:08.569804 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:08.569804 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:08.569804 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:08.569804 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:08.569804 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:08.569804 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:08.569804 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:08.569804 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 10:14:08.569804 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 10:14:08.569804 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 10:14:08.569804 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 10:14:08.570849 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:08.570849 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:08.570849 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 10:14:08.570849 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:08.570849 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 10:14:08.570849 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:08.570849 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:08.570849 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:08.570849 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:08.570849 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:08.570849 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:08.570849 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:08.570849 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:08.570849 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:08.570849 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:08.570849 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:08.570849 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:08.570849 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:08.571838 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:08.571838 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:08.571838 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 10:14:08.571838 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 10:14:08.571838 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 10:14:08.571838 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 10:14:08.571838 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 10:14:08.571838 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 10:14:08.571838 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 05 10:14:08.571838 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:08.574985 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 337/660] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 10:14:08 2022] GET /identity/v3/users/neutron => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:08.576089 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a4954c-49b9-45b3-bc78-ac721bbec26a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:08.620095 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8339d9-db65-47c1-848b-a8c7c978863b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:08.622113 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8339d9-db65-47c1-848b-a8c7c978863b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:08.710518 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a4954c-49b9-45b3-bc78-ac721bbec26a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:08.745984 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75a6cd-26bb-44ef-a76a-c67cf6aca82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:08.747192 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75a6cd-26bb-44ef-a76a-c67cf6aca82c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:08.747599 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 324/661] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:14:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 10:14:08.747907 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75a6cd-26bb-44ef-a76a-c67cf6aca82c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:08.748372 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75a6cd-26bb-44ef-a76a-c67cf6aca82c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:08.748567 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8339d9-db65-47c1-848b-a8c7c978863b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:08.756646 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb75a6cd-26bb-44ef-a76a-c67cf6aca82c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:08.758507 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb75a6cd-26bb-44ef-a76a-c67cf6aca82c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:08.770782 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-fb75a6cd-26bb-44ef-a76a-c67cf6aca82c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:08.770782 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:08.770782 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:08.770782 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:08.770782 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:08.770782 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:08.770782 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:08.770782 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:08.770782 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:08.770782 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:08.770782 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:08.770782 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:08.770782 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 10:14:08.770782 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 10:14:08.770782 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 10:14:08.771742 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 10:14:08.771742 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:08.771742 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:08.771742 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:08.771742 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:08.771742 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:08.771742 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:08.771742 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:08.771742 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:08.771742 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:08.771742 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:08.771742 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:08.771742 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:08.771742 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:08.771742 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:08.771742 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:08.771742 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:08.772723 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 10:14:08.772723 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 10:14:08.772723 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 10:14:08.772723 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 10:14:08.772723 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 10:14:08.772723 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 10:14:08.772723 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:08.772723 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:08.780030 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 338/662] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:14:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:08.781170 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75a6cd-26bb-44ef-a76a-c67cf6aca82c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:08.787529 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:08.827292 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:08.876417 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8339d9-db65-47c1-848b-a8c7c978863b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:08.898892 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75a6cd-26bb-44ef-a76a-c67cf6aca82c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:08.920498 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a98082-237f-4f24-9e0b-8aac99e9a700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:08.921553 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a98082-237f-4f24-9e0b-8aac99e9a700 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:08.921930 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a98082-237f-4f24-9e0b-8aac99e9a700 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:08.922294 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a98082-237f-4f24-9e0b-8aac99e9a700 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:08.923253 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a98082-237f-4f24-9e0b-8aac99e9a700 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:08.923656 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a98082-237f-4f24-9e0b-8aac99e9a700 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:08.925614 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a98082-237f-4f24-9e0b-8aac99e9a700 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:08.941078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f28359-fce6-4436-a9e4-5d71a2ef6522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:08.942110 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f28359-fce6-4436-a9e4-5d71a2ef6522 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:08.942547 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f28359-fce6-4436-a9e4-5d71a2ef6522 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:08.943035 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f28359-fce6-4436-a9e4-5d71a2ef6522 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:08.952708 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 325/663] 10.222.0.40 () {56 vars in 1139 bytes} [Fri Aug 5 10:14:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:08.952708 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a98082-237f-4f24-9e0b-8aac99e9a700 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:08.960125 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f28359-fce6-4436-a9e4-5d71a2ef6522 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:08.961687 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f28359-fce6-4436-a9e4-5d71a2ef6522 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:08.976414 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-36f28359-fce6-4436-a9e4-5d71a2ef6522 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:08.976414 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:08.976414 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:08.976414 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:08.976414 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:08.976414 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:08.976414 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:08.976414 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:08.976414 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:08.976414 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:08.976414 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:08.976414 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:08.976414 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:08.976414 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:08.976414 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:08.977338 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:08.977338 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:08.977338 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:08.977338 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:08.977338 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:08.977338 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:08.977338 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:08.977338 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:08.977338 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:08.977338 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:08.977338 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:08.977338 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:08.977338 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:08.977338 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:08.977338 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:08.977338 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:08.977338 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:08.978223 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:08.978223 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:08.978223 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:08.978223 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:08.982743 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 339/664] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:08.983678 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f28359-fce6-4436-a9e4-5d71a2ef6522 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:09.055764 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:09.065680 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a98082-237f-4f24-9e0b-8aac99e9a700 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:09.067547 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:09.087342 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f28359-fce6-4436-a9e4-5d71a2ef6522 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:09.102598 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f96aa-98e8-4402-811a-1e6194e5dcc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:09.103336 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f96aa-98e8-4402-811a-1e6194e5dcc6 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:09.103617 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f96aa-98e8-4402-811a-1e6194e5dcc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:09.103893 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f96aa-98e8-4402-811a-1e6194e5dcc6 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:09.104646 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587f96aa-98e8-4402-811a-1e6194e5dcc6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:09.104954 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587f96aa-98e8-4402-811a-1e6194e5dcc6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:09.106497 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587f96aa-98e8-4402-811a-1e6194e5dcc6 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:09.119131 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 326/665] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 10:14:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:09.119915 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f96aa-98e8-4402-811a-1e6194e5dcc6 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:09.121470 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af7c7d8-9efe-4678-a781-8c93d0eb5acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:09.122199 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7c7d8-9efe-4678-a781-8c93d0eb5acf None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:09.122479 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7c7d8-9efe-4678-a781-8c93d0eb5acf None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:09.122747 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7c7d8-9efe-4678-a781-8c93d0eb5acf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:09.193215 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af7c7d8-9efe-4678-a781-8c93d0eb5acf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:09.194660 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af7c7d8-9efe-4678-a781-8c93d0eb5acf None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:09.222174 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f96aa-98e8-4402-811a-1e6194e5dcc6 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:09.255487 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4ad9a1-88c6-43f5-bd90-948948633cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:09.256229 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ad9a1-88c6-43f5-bd90-948948633cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:09.256622 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ad9a1-88c6-43f5-bd90-948948633cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:09.256985 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ad9a1-88c6-43f5-bd90-948948633cf5 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:09.257764 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4ad9a1-88c6-43f5-bd90-948948633cf5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:09.258159 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4ad9a1-88c6-43f5-bd90-948948633cf5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:09.259647 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4ad9a1-88c6-43f5-bd90-948948633cf5 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:09.271541 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 327/666] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:09.272775 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a5f23-1fb0-46e3-af5d-5e6db6e9982d None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:09.273271 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a5f23-1fb0-46e3-af5d-5e6db6e9982d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:09.273375 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a5f23-1fb0-46e3-af5d-5e6db6e9982d None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:09.276020 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 328/667] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:09.276020 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f18ee24-0ae9-4867-844d-8a2b4dda4636 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:09.276239 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f18ee24-0ae9-4867-844d-8a2b4dda4636 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:09.276542 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f18ee24-0ae9-4867-844d-8a2b4dda4636 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:09.277855 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 329/668] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:09.278638 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f18ee24-0ae9-4867-844d-8a2b4dda4636 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:09.279969 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:09.294328 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:09.301209 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 340/669] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:14:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 10:14:09.302034 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af7c7d8-9efe-4678-a781-8c93d0eb5acf None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:09.380027 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f18ee24-0ae9-4867-844d-8a2b4dda4636 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:09.399505 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af7c7d8-9efe-4678-a781-8c93d0eb5acf None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:09.413229 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c63bc83-f851-4ffd-ba31-ddd721e6b9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:09.413840 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c63bc83-f851-4ffd-ba31-ddd721e6b9ad None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:09.414112 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c63bc83-f851-4ffd-ba31-ddd721e6b9ad None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:09.414388 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c63bc83-f851-4ffd-ba31-ddd721e6b9ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:09.415196 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c63bc83-f851-4ffd-ba31-ddd721e6b9ad None admin] RBAC: Adding query filter params (role.id=3a2b4319820a488198dabee34420cdc7, scope.project.id=36b8b505012d4d0bac54845b6799e743, user.id=b938f1f41db94b0eafe95bc8eeaae175) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:09.415503 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c63bc83-f851-4ffd-ba31-ddd721e6b9ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:09.416944 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c63bc83-f851-4ffd-ba31-ddd721e6b9ad None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:09.423233 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 330/670] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:14:09 2022] GET /identity/v3/role_assignments?user.id=b938f1f41db94b0eafe95bc8eeaae175&scope.project.id=36b8b505012d4d0bac54845b6799e743&role.id=3a2b4319820a488198dabee34420cdc7 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:09.424365 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216d385d-7102-4be3-81b8-a2c9ea7723aa None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:09.424643 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216d385d-7102-4be3-81b8-a2c9ea7723aa None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:09.424919 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216d385d-7102-4be3-81b8-a2c9ea7723aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:09.438726 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd137d7-ca43-486f-ac4f-3687506711b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:09.439428 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd137d7-ca43-486f-ac4f-3687506711b3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:09.439706 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd137d7-ca43-486f-ac4f-3687506711b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:09.439972 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd137d7-ca43-486f-ac4f-3687506711b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:09.448118 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd137d7-ca43-486f-ac4f-3687506711b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:09.449658 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd137d7-ca43-486f-ac4f-3687506711b3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:09.451728 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-216d385d-7102-4be3-81b8-a2c9ea7723aa None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:09.452971 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-216d385d-7102-4be3-81b8-a2c9ea7723aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:09.462568 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-dcd137d7-ca43-486f-ac4f-3687506711b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:09.462568 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:09.462568 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:09.462568 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:09.462568 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:09.462568 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:09.462568 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:09.462568 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:09.462568 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:09.462568 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:09.462568 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:09.462568 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:09.462568 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 10:14:09.462568 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 10:14:09.462568 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 10:14:09.463621 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 10:14:09.463621 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:09.463621 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:09.463621 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:09.463621 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:09.463621 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:09.463621 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:09.463621 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:09.463621 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:09.463621 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:09.463621 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:09.463621 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:09.463621 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:09.463621 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:09.463621 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:09.463621 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:09.463621 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:09.464523 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 10:14:09.464523 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 10:14:09.464523 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 10:14:09.464523 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 10:14:09.464523 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 10:14:09.464523 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 10:14:09.464523 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:09.464523 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:09.464523 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 341/671] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:14:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:09.465516 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1b607-6e39-4780-a11b-d61a79d988b3 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:09.465801 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1b607-6e39-4780-a11b-d61a79d988b3 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:09.466084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1b607-6e39-4780-a11b-d61a79d988b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:09.487481 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1cb1b607-6e39-4780-a11b-d61a79d988b3 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:09.488919 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1cb1b607-6e39-4780-a11b-d61a79d988b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:09.529984 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 331/672] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:14:09.530190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216d385d-7102-4be3-81b8-a2c9ea7723aa None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:09.530672 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:09.545225 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:09.570563 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 342/673] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:14:09.571575 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1b607-6e39-4780-a11b-d61a79d988b3 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:09.675773 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216d385d-7102-4be3-81b8-a2c9ea7723aa None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:09.684850 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1b607-6e39-4780-a11b-d61a79d988b3 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:09.714067 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a850eec-a74b-4b92-b952-641bf600c49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:09.714991 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a850eec-a74b-4b92-b952-641bf600c49d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:09.715432 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a850eec-a74b-4b92-b952-641bf600c49d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:09.715791 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a850eec-a74b-4b92-b952-641bf600c49d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:09.718404 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97873fee-c002-4ea2-ab17-1a9ebc2589e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:09.719065 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97873fee-c002-4ea2-ab17-1a9ebc2589e5 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:09.719347 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97873fee-c002-4ea2-ab17-1a9ebc2589e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:09.719618 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97873fee-c002-4ea2-ab17-1a9ebc2589e5 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:09.720361 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97873fee-c002-4ea2-ab17-1a9ebc2589e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:09.720669 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97873fee-c002-4ea2-ab17-1a9ebc2589e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:09.722118 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97873fee-c002-4ea2-ab17-1a9ebc2589e5 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:09.727042 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a850eec-a74b-4b92-b952-641bf600c49d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:09.728550 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a850eec-a74b-4b92-b952-641bf600c49d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:09.738538 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 343/674] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 10:14:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:09.738538 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97873fee-c002-4ea2-ab17-1a9ebc2589e5 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:09.748341 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-4a850eec-a74b-4b92-b952-641bf600c49d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:09.748341 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:09.748341 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:09.748341 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:09.748341 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:09.748341 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:09.748341 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:09.748341 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:09.748341 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:09.748341 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:09.748341 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:09.748341 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:09.748341 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:09.748341 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:09.748341 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:09.749443 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:09.749443 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:09.749443 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:09.749443 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:09.749443 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:09.749443 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:09.749443 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:09.749443 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:09.749443 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:09.749443 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:09.749443 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:09.749443 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:09.749443 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:09.749443 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:09.749443 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:09.749443 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:09.749443 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:09.750570 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:09.750570 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:09.750570 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:09.750570 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:09.750570 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 332/675] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:09.751132 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a850eec-a74b-4b92-b952-641bf600c49d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:09.770457 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:09.815178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:09.843677 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97873fee-c002-4ea2-ab17-1a9ebc2589e5 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:09.865515 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a850eec-a74b-4b92-b952-641bf600c49d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:09.892383 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25047215-0bf4-4b76-b0ef-aaec0f6d2a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:09.893216 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25047215-0bf4-4b76-b0ef-aaec0f6d2a06 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:09.893567 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25047215-0bf4-4b76-b0ef-aaec0f6d2a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:09.893939 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25047215-0bf4-4b76-b0ef-aaec0f6d2a06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:09.902885 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91fe614-f712-4659-8331-8c9990ad6e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:09.903623 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91fe614-f712-4659-8331-8c9990ad6e32 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:09.903902 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91fe614-f712-4659-8331-8c9990ad6e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:09.904171 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91fe614-f712-4659-8331-8c9990ad6e32 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:09.905815 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25047215-0bf4-4b76-b0ef-aaec0f6d2a06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:09.907390 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25047215-0bf4-4b76-b0ef-aaec0f6d2a06 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:09.914766 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91fe614-f712-4659-8331-8c9990ad6e32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:14:09.920667 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-d91fe614-f712-4659-8331-8c9990ad6e32 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 10:14:09.920667 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:09.920667 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:09.920667 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:09.920667 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:09.920667 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:09.920667 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:09.920667 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:09.920667 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:09.920667 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:09.920667 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:09.920667 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:09.920667 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:14:09.920667 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:14:09.920667 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:14:09.920667 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:14:09.921793 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:14:09.921793 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:14:09.921793 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:14:09.921793 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:14:09.921793 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:09.921793 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:09.921793 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:09.921793 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:09.921793 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:09.921793 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:09.921793 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:09.921793 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:09.921793 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:09.921793 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:09.921793 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:09.921793 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:09.922871 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:09.922871 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:09.922871 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:09.922871 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:09.922871 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:14:09.922871 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:14:09.922871 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:14:09.922871 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:14:09.922871 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:14:09.922871 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:14:09.922871 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 10:14:09.922871 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:09.929229 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 333/676] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 10:14:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:09.929229 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91fe614-f712-4659-8331-8c9990ad6e32 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:09.929435 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-25047215-0bf4-4b76-b0ef-aaec0f6d2a06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:09.929435 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:09.929435 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:09.929435 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:09.929435 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:09.929435 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:09.929435 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:09.929435 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:09.929435 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:09.929435 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:09.929435 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:09.929435 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:09.929435 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:09.929435 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:09.929435 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:09.930661 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:09.930661 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:09.930661 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:09.930661 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:09.930661 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:09.930661 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:09.930661 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:09.930661 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:09.930661 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:09.930661 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:09.930661 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:09.930661 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:09.930661 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:09.930661 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:09.930661 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:09.930661 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:09.930661 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:09.931881 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:09.931881 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:09.931881 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:09.931881 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:09.931881 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 344/677] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:09.931881 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25047215-0bf4-4b76-b0ef-aaec0f6d2a06 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:10.024607 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:10.036538 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91fe614-f712-4659-8331-8c9990ad6e32 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:10.042147 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25047215-0bf4-4b76-b0ef-aaec0f6d2a06 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:10.055096 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:10.074194 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ff687-0730-4217-91bc-caa6c430f17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:10.075058 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ff687-0730-4217-91bc-caa6c430f17a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:10.075230 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ff687-0730-4217-91bc-caa6c430f17a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:10.075469 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ff687-0730-4217-91bc-caa6c430f17a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:10.076262 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8ff687-0730-4217-91bc-caa6c430f17a None admin] RBAC: Adding query filter params (role.id=3a2b4319820a488198dabee34420cdc7, scope.project.id=36b8b505012d4d0bac54845b6799e743, user.id=f70369fd5b334d888de0c091d8885eb3) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:10.076575 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8ff687-0730-4217-91bc-caa6c430f17a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:10.078021 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8ff687-0730-4217-91bc-caa6c430f17a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:10.080098 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372e8b9-9f31-4de8-8d34-b02a3e575838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:10.081342 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372e8b9-9f31-4de8-8d34-b02a3e575838 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:10.082087 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372e8b9-9f31-4de8-8d34-b02a3e575838 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:10.082542 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372e8b9-9f31-4de8-8d34-b02a3e575838 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:10.083445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4372e8b9-9f31-4de8-8d34-b02a3e575838 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:10.083814 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 334/678] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:14:09 2022] GET /identity/v3/role_assignments?user.id=f70369fd5b334d888de0c091d8885eb3&scope.project.id=36b8b505012d4d0bac54845b6799e743&role.id=3a2b4319820a488198dabee34420cdc7 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:10.084111 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4372e8b9-9f31-4de8-8d34-b02a3e575838 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:10.084586 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ff687-0730-4217-91bc-caa6c430f17a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:10.086002 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4372e8b9-9f31-4de8-8d34-b02a3e575838 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:10.111777 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 345/679] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:10.113685 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372e8b9-9f31-4de8-8d34-b02a3e575838 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:10.228938 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ff687-0730-4217-91bc-caa6c430f17a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:10.264880 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372e8b9-9f31-4de8-8d34-b02a3e575838 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:10.276222 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89f4b3e-05e9-4053-8498-4b7500751f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:10.276222 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89f4b3e-05e9-4053-8498-4b7500751f64 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:10.276222 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89f4b3e-05e9-4053-8498-4b7500751f64 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:10.276222 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89f4b3e-05e9-4053-8498-4b7500751f64 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:10.276633 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89f4b3e-05e9-4053-8498-4b7500751f64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:10.276633 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89f4b3e-05e9-4053-8498-4b7500751f64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:10.278028 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89f4b3e-05e9-4053-8498-4b7500751f64 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:10.294241 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 335/680] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:10.295141 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89f4b3e-05e9-4053-8498-4b7500751f64 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:10.306052 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e2ce70-4a84-4bdc-bb08-3005a420d906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:10.306765 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2ce70-4a84-4bdc-bb08-3005a420d906 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:10.307082 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2ce70-4a84-4bdc-bb08-3005a420d906 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:10.307750 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2ce70-4a84-4bdc-bb08-3005a420d906 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:10.308480 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e2ce70-4a84-4bdc-bb08-3005a420d906 None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:10.309330 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e2ce70-4a84-4bdc-bb08-3005a420d906 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:10.310761 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e2ce70-4a84-4bdc-bb08-3005a420d906 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:10.322936 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 346/681] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 5 10:14:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:10.323092 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e2ce70-4a84-4bdc-bb08-3005a420d906 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:10.331950 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:10.353745 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:10.411170 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89f4b3e-05e9-4053-8498-4b7500751f64 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:10.447487 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e2ce70-4a84-4bdc-bb08-3005a420d906 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:10.452262 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6662b7-9885-4e52-9ab8-c1be26aa510f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:10.453082 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6662b7-9885-4e52-9ab8-c1be26aa510f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:10.453559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6662b7-9885-4e52-9ab8-c1be26aa510f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:10.453972 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6662b7-9885-4e52-9ab8-c1be26aa510f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:10.461563 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6662b7-9885-4e52-9ab8-c1be26aa510f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:10.462747 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6662b7-9885-4e52-9ab8-c1be26aa510f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:10.475028 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-5b6662b7-9885-4e52-9ab8-c1be26aa510f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:10.475028 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:10.475028 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:10.475028 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:10.475028 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:10.475028 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:10.475028 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:10.475028 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:10.475028 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:10.475028 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:10.475028 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:10.475028 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:10.475028 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 10:14:10.475028 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 10:14:10.475028 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 10:14:10.476142 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 10:14:10.476142 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:10.476142 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:10.476142 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:10.476142 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:10.476142 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:10.476142 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:10.476142 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:10.476142 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:10.476142 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:10.476142 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:10.476142 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:10.476142 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:10.476142 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:10.476142 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:10.476142 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:10.476142 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:10.477278 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 10:14:10.477278 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 10:14:10.477278 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 10:14:10.477278 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 10:14:10.477278 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 10:14:10.477278 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 10:14:10.477278 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:10.477278 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:10.479152 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 336/682] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:14:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:10.480053 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6662b7-9885-4e52-9ab8-c1be26aa510f None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:10.494719 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac16df-f98b-42f2-90af-d3eaf74b7d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:10.495398 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac16df-f98b-42f2-90af-d3eaf74b7d08 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:10.496004 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac16df-f98b-42f2-90af-d3eaf74b7d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:10.496780 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac16df-f98b-42f2-90af-d3eaf74b7d08 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:10.497638 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ac16df-f98b-42f2-90af-d3eaf74b7d08 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:10.499539 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ac16df-f98b-42f2-90af-d3eaf74b7d08 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:10.535015 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 347/683] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 5 10:14:10 2022] POST /identity/v3/users => generated 254 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:14:10.536417 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac16df-f98b-42f2-90af-d3eaf74b7d08 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:10.654161 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:10.673310 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6662b7-9885-4e52-9ab8-c1be26aa510f None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:10.677636 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:10.735509 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f49e924-4283-4299-a027-2fb534f454a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:10.736439 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f49e924-4283-4299-a027-2fb534f454a7 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:10.736866 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f49e924-4283-4299-a027-2fb534f454a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:10.737283 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f49e924-4283-4299-a027-2fb534f454a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:10.745753 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac16df-f98b-42f2-90af-d3eaf74b7d08 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:10.758187 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f49e924-4283-4299-a027-2fb534f454a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:10.759924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f49e924-4283-4299-a027-2fb534f454a7 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:10.780990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-3f49e924-4283-4299-a027-2fb534f454a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:10.780990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:10.780990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:10.780990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:10.780990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:10.780990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:10.780990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:10.780990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:10.780990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:10.780990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:10.780990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:10.780990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:10.780990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:10.780990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:10.780990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:10.782121 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:10.782121 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:10.782121 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:10.782121 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:10.782121 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:10.782121 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:10.782121 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:10.782121 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:10.782121 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:10.782121 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:10.782121 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:10.782121 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:10.782121 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:10.782121 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:10.782121 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:10.782121 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:10.782121 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:10.783298 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:10.783298 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:10.783298 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:10.783298 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:10.785520 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 337/684] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:10.791722 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f49e924-4283-4299-a027-2fb534f454a7 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:10.800963 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed632aa-021b-4766-86ff-22b95a27fbe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:10.802403 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed632aa-021b-4766-86ff-22b95a27fbe6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:10.802403 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed632aa-021b-4766-86ff-22b95a27fbe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:10.802403 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed632aa-021b-4766-86ff-22b95a27fbe6 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:10.803266 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed632aa-021b-4766-86ff-22b95a27fbe6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:10.803662 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed632aa-021b-4766-86ff-22b95a27fbe6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:10.807445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed632aa-021b-4766-86ff-22b95a27fbe6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:10.826948 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 348/685] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 10:14:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:10.830487 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed632aa-021b-4766-86ff-22b95a27fbe6 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:10.912439 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f49e924-4283-4299-a027-2fb534f454a7 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:10.943676 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed632aa-021b-4766-86ff-22b95a27fbe6 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:10.948489 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c84ff9-a1d5-4022-a496-bf402fcb2de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:10.949117 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c84ff9-a1d5-4022-a496-bf402fcb2de9 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:10.949393 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c84ff9-a1d5-4022-a496-bf402fcb2de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:10.949727 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c84ff9-a1d5-4022-a496-bf402fcb2de9 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:10.950482 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c84ff9-a1d5-4022-a496-bf402fcb2de9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:10.950805 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c84ff9-a1d5-4022-a496-bf402fcb2de9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:10.952212 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c84ff9-a1d5-4022-a496-bf402fcb2de9 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:10.970368 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 338/686] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:10.971672 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c84ff9-a1d5-4022-a496-bf402fcb2de9 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:10.972825 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:10.986496 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6274d9d-9d5b-4931-b3b4-a9a45907ed15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:10.987496 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6274d9d-9d5b-4931-b3b4-a9a45907ed15 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:10.988197 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6274d9d-9d5b-4931-b3b4-a9a45907ed15 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:10.988697 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6274d9d-9d5b-4931-b3b4-a9a45907ed15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:10.989163 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:10.991039 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6274d9d-9d5b-4931-b3b4-a9a45907ed15 None admin] RBAC: Adding query filter params (role.id=3a2b4319820a488198dabee34420cdc7, scope.project.id=36b8b505012d4d0bac54845b6799e743, user.id=a7deb94069a449269cb03406232174d3) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:10.991556 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6274d9d-9d5b-4931-b3b4-a9a45907ed15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:10.993434 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6274d9d-9d5b-4931-b3b4-a9a45907ed15 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:11.000662 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 349/687] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:14:10 2022] GET /identity/v3/role_assignments?user.id=a7deb94069a449269cb03406232174d3&scope.project.id=36b8b505012d4d0bac54845b6799e743&role.id=3a2b4319820a488198dabee34420cdc7 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:11.082799 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c84ff9-a1d5-4022-a496-bf402fcb2de9 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:11.119897 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da8602-e4ec-494b-a675-0f89732cd5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:11.120556 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da8602-e4ec-494b-a675-0f89732cd5c8 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:11.120834 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da8602-e4ec-494b-a675-0f89732cd5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:11.121104 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da8602-e4ec-494b-a675-0f89732cd5c8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:11.129580 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83da8602-e4ec-494b-a675-0f89732cd5c8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:11.131066 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83da8602-e4ec-494b-a675-0f89732cd5c8 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:11.144661 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-83da8602-e4ec-494b-a675-0f89732cd5c8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 05 10:14:11.144661 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:11.144661 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:11.144661 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:11.144661 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:11.144661 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:11.144661 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:11.144661 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:11.144661 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:11.144661 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:11.144661 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:11.144661 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:11.144661 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 10:14:11.144661 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 10:14:11.144661 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 10:14:11.144661 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 10:14:11.145672 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:11.145672 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:11.145672 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 10:14:11.145672 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:11.145672 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 10:14:11.145672 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:11.145672 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:11.145672 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:11.145672 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:11.145672 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:11.145672 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:11.145672 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:11.145672 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:11.145672 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:11.145672 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:11.145672 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:11.145672 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:11.145672 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:11.146646 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:11.146646 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:11.146646 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 10:14:11.146646 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 10:14:11.146646 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 10:14:11.146646 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 10:14:11.146646 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 10:14:11.146646 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 10:14:11.146646 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 05 10:14:11.146646 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:11.148494 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 339/688] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 10:14:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:11.157143 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6274d9d-9d5b-4931-b3b4-a9a45907ed15 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:11.271213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6274d9d-9d5b-4931-b3b4-a9a45907ed15 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:11.308115 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f429970-4b9c-4112-aee1-70ae117bc6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:11.308945 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f429970-4b9c-4112-aee1-70ae117bc6d4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:11.309321 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f429970-4b9c-4112-aee1-70ae117bc6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:11.309706 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f429970-4b9c-4112-aee1-70ae117bc6d4 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:11.310611 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f429970-4b9c-4112-aee1-70ae117bc6d4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:11.313422 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f429970-4b9c-4112-aee1-70ae117bc6d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:11.313422 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f429970-4b9c-4112-aee1-70ae117bc6d4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:11.336240 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 350/689] 10.222.0.40 () {56 vars in 1135 bytes} [Fri Aug 5 10:14:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:11.342050 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da8602-e4ec-494b-a675-0f89732cd5c8 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:11.421210 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:11.452857 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da8602-e4ec-494b-a675-0f89732cd5c8 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:11.488457 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c92a30-8d06-4068-bdcc-a1ffc5c29de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:11.489084 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c92a30-8d06-4068-bdcc-a1ffc5c29de4 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:11.489363 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c92a30-8d06-4068-bdcc-a1ffc5c29de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:11.489628 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c92a30-8d06-4068-bdcc-a1ffc5c29de4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:11.558721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44585bb-c013-406b-9ab1-3b3c176af611 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:11.559077 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44585bb-c013-406b-9ab1-3b3c176af611 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:11.559358 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44585bb-c013-406b-9ab1-3b3c176af611 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:11.561075 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 351/690] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:11.566791 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824bb0a2-a8bd-4ef8-9c69-471d42e431da None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:11.567108 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824bb0a2-a8bd-4ef8-9c69-471d42e431da None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:11.567385 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824bb0a2-a8bd-4ef8-9c69-471d42e431da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:11.587209 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-824bb0a2-a8bd-4ef8-9c69-471d42e431da None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:11.588422 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-824bb0a2-a8bd-4ef8-9c69-471d42e431da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:11.651303 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:11.686668 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 352/691] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 10:14:11.717031 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c92a30-8d06-4068-bdcc-a1ffc5c29de4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:11.718510 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c92a30-8d06-4068-bdcc-a1ffc5c29de4 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:11.720037 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824bb0a2-a8bd-4ef8-9c69-471d42e431da None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:11.876493 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:11.891314 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824bb0a2-a8bd-4ef8-9c69-471d42e431da None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:11.928771 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 340/692] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:14:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 10:14:11.946825 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c92a30-8d06-4068-bdcc-a1ffc5c29de4 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:11.961517 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60721d0e-0781-4e20-a63a-3acd79a19f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:11.962176 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60721d0e-0781-4e20-a63a-3acd79a19f1d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:11.962449 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60721d0e-0781-4e20-a63a-3acd79a19f1d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:11.962714 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60721d0e-0781-4e20-a63a-3acd79a19f1d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:11.986701 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60721d0e-0781-4e20-a63a-3acd79a19f1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:14:11.994293 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-60721d0e-0781-4e20-a63a-3acd79a19f1d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:14:11.994293 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:11.994293 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:11.994293 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:11.994293 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:11.994293 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:11.994293 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:11.994293 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:11.994293 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:11.994293 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:11.994293 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:11.994293 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:11.994293 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:14:11.994293 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:14:11.994293 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:14:11.994293 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:14:11.995307 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:14:11.995307 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:14:11.995307 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:14:11.995307 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:14:11.995307 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:11.995307 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:11.995307 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:11.995307 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:11.995307 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:11.995307 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:11.995307 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:11.995307 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:11.995307 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:11.995307 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:11.995307 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:11.995307 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:11.996238 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:11.996238 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:11.996238 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:11.996238 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:11.996238 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:14:11.996238 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:14:11.996238 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:14:11.996238 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:14:11.996238 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:14:11.996238 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:14:11.996238 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:14:11.996238 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:11.999935 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 353/693] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 10:14:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:14:12.007815 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60721d0e-0781-4e20-a63a-3acd79a19f1d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:12.073208 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:12.082893 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c92a30-8d06-4068-bdcc-a1ffc5c29de4 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:12.121254 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9353ad7-4d86-4251-8031-3efe909bf694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:12.121933 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9353ad7-4d86-4251-8031-3efe909bf694 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:12.122213 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9353ad7-4d86-4251-8031-3efe909bf694 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:12.122483 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9353ad7-4d86-4251-8031-3efe909bf694 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:12.134598 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9353ad7-4d86-4251-8031-3efe909bf694 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:12.136068 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9353ad7-4d86-4251-8031-3efe909bf694 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:12.154217 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-a9353ad7-4d86-4251-8031-3efe909bf694 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:12.154217 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:12.154217 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:12.154217 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:12.154217 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:12.154217 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:12.154217 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:12.154217 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:12.154217 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:12.154217 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:12.154217 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:12.154217 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:12.154217 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:12.154217 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:12.154217 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:12.156295 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:12.156295 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:12.156295 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:12.156295 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:12.156295 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:12.156295 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:12.156295 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:12.156295 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:12.156295 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:12.156295 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:12.156295 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:12.156295 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:12.156295 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:12.156295 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:12.156295 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:12.156295 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:12.156295 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:12.158315 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:12.158315 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:12.158315 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:12.158315 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:12.158315 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 341/694] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:12.164256 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9353ad7-4d86-4251-8031-3efe909bf694 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:12.208670 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60721d0e-0781-4e20-a63a-3acd79a19f1d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:12.218645 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:12.272069 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d34db1-39a3-4db6-88e2-299228dd5a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:12.273323 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d34db1-39a3-4db6-88e2-299228dd5a78 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:12.273751 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d34db1-39a3-4db6-88e2-299228dd5a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:12.275217 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d34db1-39a3-4db6-88e2-299228dd5a78 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:12.276088 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d34db1-39a3-4db6-88e2-299228dd5a78 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:12.277195 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d34db1-39a3-4db6-88e2-299228dd5a78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:12.278962 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d34db1-39a3-4db6-88e2-299228dd5a78 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:12.292810 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 354/695] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:14:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:12.299230 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d34db1-39a3-4db6-88e2-299228dd5a78 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:12.318868 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9353ad7-4d86-4251-8031-3efe909bf694 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:12.383585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31935445-e5d9-4341-ab05-4cc95d8a9d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:12.384506 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31935445-e5d9-4341-ab05-4cc95d8a9d72 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:12.385514 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31935445-e5d9-4341-ab05-4cc95d8a9d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:12.385928 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31935445-e5d9-4341-ab05-4cc95d8a9d72 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:12.387760 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31935445-e5d9-4341-ab05-4cc95d8a9d72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:12.389641 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31935445-e5d9-4341-ab05-4cc95d8a9d72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:12.391216 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31935445-e5d9-4341-ab05-4cc95d8a9d72 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:12.415996 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 342/696] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:12.423049 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31935445-e5d9-4341-ab05-4cc95d8a9d72 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:12.470616 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:12.477448 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d34db1-39a3-4db6-88e2-299228dd5a78 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:12.514527 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed465235-0b5d-43fe-a693-e652b6becf98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:12.515597 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed465235-0b5d-43fe-a693-e652b6becf98 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:12.515933 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed465235-0b5d-43fe-a693-e652b6becf98 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:12.516272 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed465235-0b5d-43fe-a693-e652b6becf98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:12.527343 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed465235-0b5d-43fe-a693-e652b6becf98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:12.528980 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed465235-0b5d-43fe-a693-e652b6becf98 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:12.543701 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-ed465235-0b5d-43fe-a693-e652b6becf98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:12.543701 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:12.543701 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:12.543701 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:12.543701 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:12.543701 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:12.543701 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:12.543701 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:12.543701 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:12.543701 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:12.543701 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:12.543701 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:12.543701 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:12.543701 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:12.543701 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:12.544721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:12.544721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:12.544721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:12.544721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:12.544721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:12.544721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:12.544721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:12.544721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:12.544721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:12.544721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:12.544721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:12.544721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:12.544721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:12.544721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:12.544721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:12.544721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:12.544721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:12.545999 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:12.545999 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:12.545999 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:12.545999 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:12.545999 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 355/697] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:14:12.546455 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1a2b8-76e9-41cd-9c18-592676d4aa63 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:12.546602 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1a2b8-76e9-41cd-9c18-592676d4aa63 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:12.547572 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1a2b8-76e9-41cd-9c18-592676d4aa63 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:12.553191 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 356/698] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:12.553919 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb1a2b8-76e9-41cd-9c18-592676d4aa63 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:12.558316 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:12.568618 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31935445-e5d9-4341-ab05-4cc95d8a9d72 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:12.606758 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00450b2d-5251-4a74-9056-aac000715da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:12.607832 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00450b2d-5251-4a74-9056-aac000715da4 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:12.608304 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00450b2d-5251-4a74-9056-aac000715da4 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:12.608762 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00450b2d-5251-4a74-9056-aac000715da4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:12.616548 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00450b2d-5251-4a74-9056-aac000715da4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:12.618444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00450b2d-5251-4a74-9056-aac000715da4 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:12.632311 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-00450b2d-5251-4a74-9056-aac000715da4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:12.632311 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:12.632311 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:12.632311 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:12.632311 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:12.632311 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:12.632311 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:12.632311 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:12.632311 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:12.632311 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:12.632311 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:12.632311 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:12.632311 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 10:14:12.632311 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 10:14:12.632311 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 10:14:12.633317 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 10:14:12.633317 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:12.633317 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:12.633317 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:12.633317 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:12.633317 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:12.633317 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:12.633317 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:12.633317 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:12.633317 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:12.633317 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:12.633317 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:12.633317 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:12.633317 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:12.633317 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:12.633317 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:12.633317 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:12.634220 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 10:14:12.634220 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 10:14:12.634220 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 10:14:12.634220 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 10:14:12.634220 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 10:14:12.634220 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 10:14:12.634220 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:12.634220 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:12.636804 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 343/699] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:14:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:12.638360 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8482a-c094-4e74-86c1-aaafee33059c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:12.638849 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8482a-c094-4e74-86c1-aaafee33059c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:12.639349 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8482a-c094-4e74-86c1-aaafee33059c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:12.663759 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-6be8482a-c094-4e74-86c1-aaafee33059c None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:12.664955 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6be8482a-c094-4e74-86c1-aaafee33059c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:12.681577 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb1a2b8-76e9-41cd-9c18-592676d4aa63 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:12.718416 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a6ecf-c321-4500-b3c4-290d458c8a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:12.719467 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a6ecf-c321-4500-b3c4-290d458c8a25 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:12.719970 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a6ecf-c321-4500-b3c4-290d458c8a25 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:12.720438 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a6ecf-c321-4500-b3c4-290d458c8a25 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:12.721323 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4a6ecf-c321-4500-b3c4-290d458c8a25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:12.721813 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4a6ecf-c321-4500-b3c4-290d458c8a25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:12.723554 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4a6ecf-c321-4500-b3c4-290d458c8a25 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:12.739572 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 357/700] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:12.740914 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a6ecf-c321-4500-b3c4-290d458c8a25 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:12.752680 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:12.760595 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 344/701] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:14:12.761766 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be8482a-c094-4e74-86c1-aaafee33059c None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:12.844392 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:12.848987 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a6ecf-c321-4500-b3c4-290d458c8a25 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:12.868624 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be8482a-c094-4e74-86c1-aaafee33059c None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:12.886735 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39182bb3-8389-45f2-9f4c-2e506f4b26ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:12.887919 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39182bb3-8389-45f2-9f4c-2e506f4b26ae None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:12.888475 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39182bb3-8389-45f2-9f4c-2e506f4b26ae None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:12.888939 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39182bb3-8389-45f2-9f4c-2e506f4b26ae None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:12.889949 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39182bb3-8389-45f2-9f4c-2e506f4b26ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:12.890444 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39182bb3-8389-45f2-9f4c-2e506f4b26ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:12.892105 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39182bb3-8389-45f2-9f4c-2e506f4b26ae None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:12.903283 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e064d625-cef4-463f-a68c-0f4db18b8fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:12.903929 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e064d625-cef4-463f-a68c-0f4db18b8fc2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:12.904207 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e064d625-cef4-463f-a68c-0f4db18b8fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:12.904469 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e064d625-cef4-463f-a68c-0f4db18b8fc2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:12.911144 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 358/702] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 10:14:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:12.911969 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e064d625-cef4-463f-a68c-0f4db18b8fc2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:12.912715 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39182bb3-8389-45f2-9f4c-2e506f4b26ae None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:12.913411 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e064d625-cef4-463f-a68c-0f4db18b8fc2 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:12.925964 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-e064d625-cef4-463f-a68c-0f4db18b8fc2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 10:14:12.925964 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:12.925964 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:12.925964 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:12.925964 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:12.925964 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:12.925964 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:12.925964 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:12.925964 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:12.925964 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:12.925964 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:12.925964 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:12.925964 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 10:14:12.925964 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 10:14:12.925964 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 10:14:12.925964 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 10:14:12.927041 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:12.927041 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:12.927041 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 10:14:12.927041 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:12.927041 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 10:14:12.927041 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:12.927041 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:12.927041 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:12.927041 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:12.927041 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:12.927041 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:12.927041 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:12.927041 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:12.927041 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:12.927041 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:12.927041 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:12.927041 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:12.927041 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:12.928474 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:12.928474 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:12.928474 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 10:14:12.928474 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 10:14:12.928474 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 10:14:12.928474 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 10:14:12.928474 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 10:14:12.928474 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 10:14:12.928474 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 10:14:12.928474 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:12.931250 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 345/703] 10.222.0.40 () {56 vars in 1093 bytes} [Fri Aug 5 10:14:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:12.932125 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e064d625-cef4-463f-a68c-0f4db18b8fc2 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:12.989963 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:13.030367 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39182bb3-8389-45f2-9f4c-2e506f4b26ae None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:13.035150 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e064d625-cef4-463f-a68c-0f4db18b8fc2 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:13.068225 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f09ef6-6dc0-48c0-a2c3-4a487e5bf36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:13.069395 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f09ef6-6dc0-48c0-a2c3-4a487e5bf36b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:13.069773 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f09ef6-6dc0-48c0-a2c3-4a487e5bf36b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:13.070256 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f09ef6-6dc0-48c0-a2c3-4a487e5bf36b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:13.071045 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7db134e-a2aa-46ef-a0ec-0ea0ef3147e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:13.071192 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f09ef6-6dc0-48c0-a2c3-4a487e5bf36b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:13.071932 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7db134e-a2aa-46ef-a0ec-0ea0ef3147e9 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:13.072346 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7db134e-a2aa-46ef-a0ec-0ea0ef3147e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:13.072752 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7db134e-a2aa-46ef-a0ec-0ea0ef3147e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:13.072891 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f09ef6-6dc0-48c0-a2c3-4a487e5bf36b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:13.073807 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7db134e-a2aa-46ef-a0ec-0ea0ef3147e9 None admin] RBAC: Adding query filter params (role.id=3a2b4319820a488198dabee34420cdc7, scope.project.id=36b8b505012d4d0bac54845b6799e743, user.id=c80cf0e429f04f229aea1aa9293a93b2) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:13.074260 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7db134e-a2aa-46ef-a0ec-0ea0ef3147e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:13.076205 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7db134e-a2aa-46ef-a0ec-0ea0ef3147e9 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:13.079447 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:13.083575 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 346/704] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:14:12 2022] GET /identity/v3/role_assignments?user.id=c80cf0e429f04f229aea1aa9293a93b2&scope.project.id=36b8b505012d4d0bac54845b6799e743&role.id=3a2b4319820a488198dabee34420cdc7 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:13.085018 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7db134e-a2aa-46ef-a0ec-0ea0ef3147e9 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:13.094164 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-a5f09ef6-6dc0-48c0-a2c3-4a487e5bf36b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 10:14:13.094164 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:13.094164 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 10:14:13.094164 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 10:14:13.094164 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 10:14:13.094164 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 10:14:13.094164 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 10:14:13.094164 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:13.094164 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 10:14:13.094164 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:13.094164 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 10:14:13.094164 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:13.094164 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:13.094164 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:13.094164 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:13.094164 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:13.095353 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:13.095353 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:13.095353 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:13.095353 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:13.095353 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:13.095353 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:13.095353 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:13.095353 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 10:14:13.095353 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 10:14:13.095353 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 10:14:13.095353 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 10:14:13.095353 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:13.095353 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:13.095353 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:13.095353 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:13.095353 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:13.096396 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:13.096396 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:13.096396 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:13.096396 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:13.096396 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:13.096396 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:13.096396 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:13.096396 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:13.096396 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:13.096396 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:13.096396 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:13.096396 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 10:14:13.096396 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:13.096396 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 10:14:13.096396 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:13.099226 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 359/705] 10.222.0.40 () {56 vars in 1129 bytes} [Fri Aug 5 10:14:12 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:13.108141 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f09ef6-6dc0-48c0-a2c3-4a487e5bf36b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:13.202991 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7db134e-a2aa-46ef-a0ec-0ea0ef3147e9 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:13.240343 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882821af-8cac-4eda-b1c1-4aaeccb848b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:13.241368 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882821af-8cac-4eda-b1c1-4aaeccb848b6 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:13.241861 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882821af-8cac-4eda-b1c1-4aaeccb848b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:13.242336 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882821af-8cac-4eda-b1c1-4aaeccb848b6 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:13.243392 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882821af-8cac-4eda-b1c1-4aaeccb848b6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:13.243913 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882821af-8cac-4eda-b1c1-4aaeccb848b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:13.245577 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882821af-8cac-4eda-b1c1-4aaeccb848b6 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:13.265242 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 347/706] 10.222.0.40 () {56 vars in 1133 bytes} [Fri Aug 5 10:14:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:13.271035 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882821af-8cac-4eda-b1c1-4aaeccb848b6 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:13.323920 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f09ef6-6dc0-48c0-a2c3-4a487e5bf36b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:13.334924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:13.359813 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:13.367063 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6129fb21-142e-4ef4-a2c0-81290b21d52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:13.367564 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6129fb21-142e-4ef4-a2c0-81290b21d52b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:13.367848 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6129fb21-142e-4ef4-a2c0-81290b21d52b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:13.368122 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6129fb21-142e-4ef4-a2c0-81290b21d52b None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:13.368959 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6129fb21-142e-4ef4-a2c0-81290b21d52b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:13.369221 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6129fb21-142e-4ef4-a2c0-81290b21d52b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:13.370710 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6129fb21-142e-4ef4-a2c0-81290b21d52b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:13.376686 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 360/707] 10.222.0.40 () {56 vars in 1124 bytes} [Fri Aug 5 10:14:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:13.381361 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882821af-8cac-4eda-b1c1-4aaeccb848b6 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:13.382087 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6129fb21-142e-4ef4-a2c0-81290b21d52b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:13.423506 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f0c12-c76e-411a-bd76-4a5d50378b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:13.424757 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f0c12-c76e-411a-bd76-4a5d50378b4f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:13.425474 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f0c12-c76e-411a-bd76-4a5d50378b4f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:13.426350 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f0c12-c76e-411a-bd76-4a5d50378b4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:13.512945 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591f0c12-c76e-411a-bd76-4a5d50378b4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:13.514584 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591f0c12-c76e-411a-bd76-4a5d50378b4f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:13.518092 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6129fb21-142e-4ef4-a2c0-81290b21d52b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:13.565220 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e863be-35df-48d0-9608-f17c7218104c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:13.566095 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e863be-35df-48d0-9608-f17c7218104c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:13.566372 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e863be-35df-48d0-9608-f17c7218104c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:13.567021 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e863be-35df-48d0-9608-f17c7218104c None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:13.567714 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e863be-35df-48d0-9608-f17c7218104c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:13.568185 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e863be-35df-48d0-9608-f17c7218104c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:13.570115 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e863be-35df-48d0-9608-f17c7218104c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:13.575659 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:13.579203 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 361/708] 10.222.0.40 () {56 vars in 1124 bytes} [Fri Aug 5 10:14:13 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:13.581326 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa5546a-e6b0-49a8-ab36-28277b6b25ec None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:13.581932 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa5546a-e6b0-49a8-ab36-28277b6b25ec None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:13.582337 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa5546a-e6b0-49a8-ab36-28277b6b25ec None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:13.584071 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 362/709] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:13.590196 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924277d-a5cc-4ce5-a07d-9c375b9e9833 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:13.591142 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924277d-a5cc-4ce5-a07d-9c375b9e9833 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:13.591426 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924277d-a5cc-4ce5-a07d-9c375b9e9833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:13.629704 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-0924277d-a5cc-4ce5-a07d-9c375b9e9833 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:13.631004 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0924277d-a5cc-4ce5-a07d-9c375b9e9833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:13.659241 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 348/710] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:14:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 10:14:13.664912 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f0c12-c76e-411a-bd76-4a5d50378b4f None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:13.727432 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:13.741553 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 363/711] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:14:13.743316 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e148719c-0895-4d9e-83fe-186f41af425d None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:13.743796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e148719c-0895-4d9e-83fe-186f41af425d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:13.744261 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e148719c-0895-4d9e-83fe-186f41af425d None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:13.749967 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 364/712] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:13.751680 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51c79a-2408-47e4-b581-48aaa567d1b2 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:13.752159 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51c79a-2408-47e4-b581-48aaa567d1b2 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:13.752658 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51c79a-2408-47e4-b581-48aaa567d1b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:13.775075 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1e51c79a-2408-47e4-b581-48aaa567d1b2 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:13.776577 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1e51c79a-2408-47e4-b581-48aaa567d1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:13.809912 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f0c12-c76e-411a-bd76-4a5d50378b4f None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:13.852680 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d51b309-6b92-4554-bc9a-8e176b98806e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:13.852919 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 365/713] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:14:13.857952 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51b309-6b92-4554-bc9a-8e176b98806e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:13.858097 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e51c79a-2408-47e4-b581-48aaa567d1b2 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:13.860280 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51b309-6b92-4554-bc9a-8e176b98806e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:13.861868 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51b309-6b92-4554-bc9a-8e176b98806e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:13.885763 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d51b309-6b92-4554-bc9a-8e176b98806e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:13.888528 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d51b309-6b92-4554-bc9a-8e176b98806e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:13.908294 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-1d51b309-6b92-4554-bc9a-8e176b98806e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:13.908294 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:13.908294 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:13.908294 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:13.908294 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:13.908294 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:13.908294 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:13.908294 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:13.908294 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:13.908294 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:13.908294 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:13.908294 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:13.908294 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:13.908294 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:13.908294 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:13.909291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:13.909291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:13.909291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:13.909291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:13.909291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:13.909291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:13.909291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:13.909291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:13.909291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:13.909291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:13.909291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:13.909291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:13.909291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:13.909291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:13.909291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:13.909291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:13.909291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:13.910165 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:13.910165 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:13.910165 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:13.910165 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:13.910165 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 349/714] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:14:13.910877 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d51b309-6b92-4554-bc9a-8e176b98806e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:13.935423 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:13.981163 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e51c79a-2408-47e4-b581-48aaa567d1b2 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:14.004479 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:14.021338 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae928afe-74c8-4cfb-b146-2233251e3b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:14.022276 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae928afe-74c8-4cfb-b146-2233251e3b0d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:14.022769 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae928afe-74c8-4cfb-b146-2233251e3b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:14.023275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae928afe-74c8-4cfb-b146-2233251e3b0d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:14.034637 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae928afe-74c8-4cfb-b146-2233251e3b0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:14:14.039912 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-ae928afe-74c8-4cfb-b146-2233251e3b0d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 10:14:14.039912 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:14.039912 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:14.039912 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:14.039912 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:14.039912 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:14.039912 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:14.039912 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:14.039912 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:14.039912 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:14.039912 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:14.039912 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:14.039912 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:14:14.039912 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:14:14.039912 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:14:14.039912 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:14:14.040812 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:14:14.040812 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:14:14.040812 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:14:14.040812 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:14:14.040812 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:14.040812 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:14.040812 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:14.040812 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:14.040812 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:14.040812 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:14.040812 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:14.040812 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:14.040812 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:14.040812 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:14.040812 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:14.040812 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:14.041720 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:14.041720 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:14.041720 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:14.041720 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:14.041720 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:14:14.041720 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:14:14.041720 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:14:14.041720 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:14:14.041720 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:14:14.041720 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:14:14.041720 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 10:14:14.041720 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:14.044803 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 366/715] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 10:14:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:14.045805 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae928afe-74c8-4cfb-b146-2233251e3b0d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:14.052100 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d51b309-6b92-4554-bc9a-8e176b98806e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:14.097649 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320441fc-4a3b-4e19-b140-9d81a5df5b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:14.098287 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320441fc-4a3b-4e19-b140-9d81a5df5b3a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:14.098572 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320441fc-4a3b-4e19-b140-9d81a5df5b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:14.098846 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320441fc-4a3b-4e19-b140-9d81a5df5b3a None admin] PATH_INFO: `/v3/services/network` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:14.099511 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320441fc-4a3b-4e19-b140-9d81a5df5b3a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:14.101185 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320441fc-4a3b-4e19-b140-9d81a5df5b3a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:14.116831 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-320441fc-4a3b-4e19-b140-9d81a5df5b3a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 10:14:14.116831 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:14.116831 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 10:14:14.116831 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 10:14:14.116831 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 10:14:14.116831 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 10:14:14.116831 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 10:14:14.116831 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:14.116831 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 10:14:14.116831 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:14.116831 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 10:14:14.116831 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:14.116831 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:14.116831 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:14.116831 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:14.116831 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:14.116831 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:14.117893 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:14.117893 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:14.117893 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:14.117893 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:14.117893 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:14.117893 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:14.117893 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 10:14:14.117893 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 10:14:14.117893 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 10:14:14.117893 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 10:14:14.117893 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:14.117893 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:14.117893 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:14.117893 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:14.117893 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:14.117893 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:14.118728 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:14.118728 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:14.118728 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:14.118728 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:14.118728 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:14.118728 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:14.118728 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:14.118728 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:14.118728 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:14.118728 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:14.118728 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 10:14:14.118728 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:14.118728 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 10:14:14.118728 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:14.123009 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 350/716] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:14:13 2022] GET /identity/v3/services/network => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:14.125137 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320441fc-4a3b-4e19-b140-9d81a5df5b3a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:14.167480 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae928afe-74c8-4cfb-b146-2233251e3b0d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:14.205768 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590c4038-3698-46d3-b023-af8cd007aeb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:14.206769 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c4038-3698-46d3-b023-af8cd007aeb7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:14.207282 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c4038-3698-46d3-b023-af8cd007aeb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:14.207770 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c4038-3698-46d3-b023-af8cd007aeb7 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:14.208742 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590c4038-3698-46d3-b023-af8cd007aeb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:14.209270 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590c4038-3698-46d3-b023-af8cd007aeb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:14.211634 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590c4038-3698-46d3-b023-af8cd007aeb7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:14.225163 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 367/717] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:14.225982 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590c4038-3698-46d3-b023-af8cd007aeb7 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:14.236951 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:14.270007 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320441fc-4a3b-4e19-b140-9d81a5df5b3a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:14.279316 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:14.308005 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59762fb8-c0a4-4edc-8cb7-319a3c0d1bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:14.308664 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59762fb8-c0a4-4edc-8cb7-319a3c0d1bc8 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:14.309042 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59762fb8-c0a4-4edc-8cb7-319a3c0d1bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:14.309246 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59762fb8-c0a4-4edc-8cb7-319a3c0d1bc8 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:14.309976 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59762fb8-c0a4-4edc-8cb7-319a3c0d1bc8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:14.310282 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59762fb8-c0a4-4edc-8cb7-319a3c0d1bc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:14.311701 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59762fb8-c0a4-4edc-8cb7-319a3c0d1bc8 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:14.320536 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 351/718] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 5 10:14:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:14.321277 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59762fb8-c0a4-4edc-8cb7-319a3c0d1bc8 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:14.365347 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590c4038-3698-46d3-b023-af8cd007aeb7 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:14.405580 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd5a7c5-73d6-4a06-82b2-3fe8ca7be3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:14.406585 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5a7c5-73d6-4a06-82b2-3fe8ca7be3ee None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:14.407203 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5a7c5-73d6-4a06-82b2-3fe8ca7be3ee None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:14.407670 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5a7c5-73d6-4a06-82b2-3fe8ca7be3ee None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:14.408656 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd5a7c5-73d6-4a06-82b2-3fe8ca7be3ee None admin] RBAC: Adding query filter params (name=network) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:14.409156 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd5a7c5-73d6-4a06-82b2-3fe8ca7be3ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:14.410812 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd5a7c5-73d6-4a06-82b2-3fe8ca7be3ee None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:14.421119 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 368/719] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 5 10:14:14 2022] GET /identity/v3/services?name=network => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:14.422242 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd5a7c5-73d6-4a06-82b2-3fe8ca7be3ee None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:14.426864 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59762fb8-c0a4-4edc-8cb7-319a3c0d1bc8 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:14.463829 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e97556-78d6-4b12-b65f-55b099365c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:14.464484 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e97556-78d6-4b12-b65f-55b099365c2a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:14.464769 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e97556-78d6-4b12-b65f-55b099365c2a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:14.465190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e97556-78d6-4b12-b65f-55b099365c2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:14.468306 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:14.473671 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e97556-78d6-4b12-b65f-55b099365c2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:14.476161 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e97556-78d6-4b12-b65f-55b099365c2a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:14.492559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-f1e97556-78d6-4b12-b65f-55b099365c2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:14.492559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:14.492559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:14.492559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:14.492559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:14.492559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:14.492559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:14.492559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:14.492559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:14.492559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:14.492559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:14.492559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:14.492559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 10:14:14.492559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 10:14:14.492559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 10:14:14.493719 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 10:14:14.493719 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:14.493719 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:14.493719 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:14.493719 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:14.493719 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:14.493719 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:14.493719 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:14.493719 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:14.493719 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:14.493719 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:14.493719 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:14.493719 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:14.493719 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:14.493719 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:14.493719 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:14.493719 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:14.494841 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 10:14:14.494841 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 10:14:14.494841 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 10:14:14.494841 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 10:14:14.494841 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 10:14:14.494841 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 10:14:14.494841 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:14.494841 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:14.495639 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 352/720] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:14:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:14.496789 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e97556-78d6-4b12-b65f-55b099365c2a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:14.558360 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd5a7c5-73d6-4a06-82b2-3fe8ca7be3ee None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:14.562151 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:14.609586 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61268db5-e90d-4d12-a7da-fc0dcd0a00e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:14.610240 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61268db5-e90d-4d12-a7da-fc0dcd0a00e3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:14.610519 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61268db5-e90d-4d12-a7da-fc0dcd0a00e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:14.610791 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61268db5-e90d-4d12-a7da-fc0dcd0a00e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:14.611055 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e97556-78d6-4b12-b65f-55b099365c2a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:14.625322 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61268db5-e90d-4d12-a7da-fc0dcd0a00e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:14.627824 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61268db5-e90d-4d12-a7da-fc0dcd0a00e3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:14.644582 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-61268db5-e90d-4d12-a7da-fc0dcd0a00e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:14.644582 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:14.644582 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:14.644582 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:14.644582 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:14.644582 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:14.644582 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:14.644582 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:14.644582 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:14.644582 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:14.644582 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:14.644582 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:14.644582 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:14.644582 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:14.644582 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:14.645513 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:14.645513 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:14.645513 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:14.645513 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:14.645513 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:14.645513 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:14.645513 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:14.645513 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:14.645513 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:14.645513 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:14.645513 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:14.645513 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:14.645513 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:14.645513 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:14.645513 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:14.645513 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:14.645513 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:14.646389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:14.646389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:14.646389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:14.646389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:14.646389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 369/721] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:14.647122 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61268db5-e90d-4d12-a7da-fc0dcd0a00e3 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:14.654256 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9620e5-ef16-4d58-b062-10402208b360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:14.655208 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9620e5-ef16-4d58-b062-10402208b360 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:14.655704 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9620e5-ef16-4d58-b062-10402208b360 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:14.656168 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9620e5-ef16-4d58-b062-10402208b360 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:14.657048 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9620e5-ef16-4d58-b062-10402208b360 None admin] RBAC: Adding query filter params (type=network) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:14.658078 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9620e5-ef16-4d58-b062-10402208b360 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:14.660114 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9620e5-ef16-4d58-b062-10402208b360 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:14.669057 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 353/722] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 5 10:14:14 2022] GET /identity/v3/services?type=network => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:14.670124 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9620e5-ef16-4d58-b062-10402208b360 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:14.745508 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:14.841206 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9620e5-ef16-4d58-b062-10402208b360 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:14.924868 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f7423-f1c3-4d8d-a8f2-cf0e13076a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:14.925522 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f7423-f1c3-4d8d-a8f2-cf0e13076a22 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:14.925802 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f7423-f1c3-4d8d-a8f2-cf0e13076a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:14.926075 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f7423-f1c3-4d8d-a8f2-cf0e13076a22 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:14.926786 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041f7423-f1c3-4d8d-a8f2-cf0e13076a22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:14.927114 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041f7423-f1c3-4d8d-a8f2-cf0e13076a22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:14.928504 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041f7423-f1c3-4d8d-a8f2-cf0e13076a22 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:14.955107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 354/723] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:14.961203 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f7423-f1c3-4d8d-a8f2-cf0e13076a22 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:15.093298 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61268db5-e90d-4d12-a7da-fc0dcd0a00e3 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:15.128522 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3bbe94-d96b-414a-b4e2-72181706b6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:15.129152 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3bbe94-d96b-414a-b4e2-72181706b6f0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:15.129433 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3bbe94-d96b-414a-b4e2-72181706b6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:15.129707 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3bbe94-d96b-414a-b4e2-72181706b6f0 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:15.130463 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3bbe94-d96b-414a-b4e2-72181706b6f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:15.130773 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3bbe94-d96b-414a-b4e2-72181706b6f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:15.132371 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3bbe94-d96b-414a-b4e2-72181706b6f0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:15.147163 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 370/724] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 10:14:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 500 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:15.157565 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3bbe94-d96b-414a-b4e2-72181706b6f0 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:15.163612 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f7423-f1c3-4d8d-a8f2-cf0e13076a22 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:15.207277 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:15.247550 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:15.299637 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa9aa3-95b5-4a46-9967-3e73654a8635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:15.300312 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa9aa3-95b5-4a46-9967-3e73654a8635 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:15.300594 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa9aa3-95b5-4a46-9967-3e73654a8635 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:15.300869 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa9aa3-95b5-4a46-9967-3e73654a8635 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:15.318158 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa9aa3-95b5-4a46-9967-3e73654a8635 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:15.327805 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa9aa3-95b5-4a46-9967-3e73654a8635 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:15.342367 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3bbe94-d96b-414a-b4e2-72181706b6f0 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:15.358461 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-eafa9aa3-95b5-4a46-9967-3e73654a8635 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 10:14:15.358461 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:15.358461 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:15.358461 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:15.358461 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:15.358461 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:15.358461 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:15.358461 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:15.358461 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:15.358461 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:15.358461 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:15.358461 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:15.358461 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 10:14:15.358461 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 10:14:15.358461 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 10:14:15.358461 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 10:14:15.359508 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:15.359508 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:15.359508 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 10:14:15.359508 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:15.359508 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 10:14:15.359508 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:15.359508 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:15.359508 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:15.359508 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:15.359508 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:15.359508 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:15.359508 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:15.359508 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:15.359508 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:15.359508 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:15.359508 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:15.359508 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:15.359508 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:15.360428 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:15.360428 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:15.360428 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 10:14:15.360428 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 10:14:15.360428 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 10:14:15.360428 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 10:14:15.360428 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 10:14:15.360428 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 10:14:15.360428 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 10:14:15.360428 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:15.365546 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 355/725] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Aug 5 10:14:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 405 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:14:15.371440 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa9aa3-95b5-4a46-9967-3e73654a8635 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:15.389089 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9846aa22-561e-4a6b-a852-7dadd8bcfab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:15.390519 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9846aa22-561e-4a6b-a852-7dadd8bcfab2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:15.391477 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9846aa22-561e-4a6b-a852-7dadd8bcfab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:15.391989 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9846aa22-561e-4a6b-a852-7dadd8bcfab2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:15.393247 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9846aa22-561e-4a6b-a852-7dadd8bcfab2 None admin] RBAC: Adding query filter params (role.id=daef89adcaeb48a382a0bcb03340ccbb, scope.project.id=36b8b505012d4d0bac54845b6799e743, user.id=b938f1f41db94b0eafe95bc8eeaae175) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:15.393760 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9846aa22-561e-4a6b-a852-7dadd8bcfab2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:15.395446 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9846aa22-561e-4a6b-a852-7dadd8bcfab2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:15.407003 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 371/726] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:14:15 2022] GET /identity/v3/role_assignments?user.id=b938f1f41db94b0eafe95bc8eeaae175&scope.project.id=36b8b505012d4d0bac54845b6799e743&role.id=daef89adcaeb48a382a0bcb03340ccbb => generated 259 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:15.533385 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41878656-7244-4c8c-a03e-317490a58e45 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:15.533753 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41878656-7244-4c8c-a03e-317490a58e45 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:15.533947 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41878656-7244-4c8c-a03e-317490a58e45 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:15.539362 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 372/727] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 10:14:15.548168 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148cb50-a258-40ee-9cf4-378fe715946b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:15.548576 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148cb50-a258-40ee-9cf4-378fe715946b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:15.548710 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148cb50-a258-40ee-9cf4-378fe715946b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:15.559454 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa9aa3-95b5-4a46-9967-3e73654a8635 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:15.593559 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a148cb50-a258-40ee-9cf4-378fe715946b None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:15.593559 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a148cb50-a258-40ee-9cf4-378fe715946b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:15.595581 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b6922f-702d-47a4-a4e2-ad094f519d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:15.596288 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6922f-702d-47a4-a4e2-ad094f519d68 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:15.596633 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6922f-702d-47a4-a4e2-ad094f519d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:15.596992 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6922f-702d-47a4-a4e2-ad094f519d68 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:15.597872 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b6922f-702d-47a4-a4e2-ad094f519d68 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:15.598250 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b6922f-702d-47a4-a4e2-ad094f519d68 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:15.599828 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b6922f-702d-47a4-a4e2-ad094f519d68 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:15.619621 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 356/728] 10.222.0.40 () {56 vars in 1149 bytes} [Fri Aug 5 10:14:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:15.629600 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b6922f-702d-47a4-a4e2-ad094f519d68 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:15.685406 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:15.753632 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 373/729] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 10:14:15.767132 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b6922f-702d-47a4-a4e2-ad094f519d68 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:15.787788 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a148cb50-a258-40ee-9cf4-378fe715946b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:15.802739 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:15.806278 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a66b18-b45f-4ac8-8c5e-5f95ccedce8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:15.806934 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a66b18-b45f-4ac8-8c5e-5f95ccedce8d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:15.807214 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a66b18-b45f-4ac8-8c5e-5f95ccedce8d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:15.807491 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a66b18-b45f-4ac8-8c5e-5f95ccedce8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:15.900971 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a66b18-b45f-4ac8-8c5e-5f95ccedce8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:15.903965 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a66b18-b45f-4ac8-8c5e-5f95ccedce8d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:15.906599 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a148cb50-a258-40ee-9cf4-378fe715946b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:15.945277 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24968df3-663e-4e5f-8d52-0a93787c7d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:15.946225 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24968df3-663e-4e5f-8d52-0a93787c7d6a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:15.946609 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24968df3-663e-4e5f-8d52-0a93787c7d6a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:15.947014 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24968df3-663e-4e5f-8d52-0a93787c7d6a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:15.960025 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24968df3-663e-4e5f-8d52-0a93787c7d6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:14:15.965300 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-24968df3-663e-4e5f-8d52-0a93787c7d6a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:14:15.965300 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:15.965300 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:15.965300 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:15.965300 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:15.965300 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:15.965300 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:15.965300 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:15.965300 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:15.965300 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:15.965300 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:15.965300 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:15.965300 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:14:15.965300 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:14:15.965300 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:14:15.965300 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:14:15.966601 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:14:15.966601 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:14:15.966601 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:14:15.966601 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:14:15.966601 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:15.966601 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:15.966601 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:15.966601 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:15.966601 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:15.966601 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:15.966601 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:15.966601 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:15.966601 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:15.966601 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:15.966601 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:15.966601 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:15.967798 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:15.967798 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:15.967798 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:15.967798 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:15.967798 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:14:15.967798 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:14:15.967798 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:14:15.967798 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:14:15.967798 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:14:15.967798 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:14:15.967798 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:14:15.967798 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:15.967798 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 374/730] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 10:14:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:14:15.968655 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5436bf6f-b8ba-4c49-ac8f-98d9766e0742 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:15.968864 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5436bf6f-b8ba-4c49-ac8f-98d9766e0742 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:15.969138 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5436bf6f-b8ba-4c49-ac8f-98d9766e0742 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:15.970665 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 375/731] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:15.971260 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:15.972857 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5436bf6f-b8ba-4c49-ac8f-98d9766e0742 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:16.033105 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 357/732] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:14:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 10:14:16.034447 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbfcca3-e4b4-495d-82b4-ce57dc72a658 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:16.034731 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbfcca3-e4b4-495d-82b4-ce57dc72a658 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:16.035027 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbfcca3-e4b4-495d-82b4-ce57dc72a658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:16.057167 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-5fbfcca3-e4b4-495d-82b4-ce57dc72a658 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:16.058582 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5fbfcca3-e4b4-495d-82b4-ce57dc72a658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:16.063809 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:16.106553 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5436bf6f-b8ba-4c49-ac8f-98d9766e0742 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:16.141667 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19ada83-e8c6-4988-a0f1-42b2b8cd8d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:16.142291 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ada83-e8c6-4988-a0f1-42b2b8cd8d4f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:16.142567 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ada83-e8c6-4988-a0f1-42b2b8cd8d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:16.142837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ada83-e8c6-4988-a0f1-42b2b8cd8d4f None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:16.143583 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19ada83-e8c6-4988-a0f1-42b2b8cd8d4f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:16.143884 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19ada83-e8c6-4988-a0f1-42b2b8cd8d4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:16.145264 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19ada83-e8c6-4988-a0f1-42b2b8cd8d4f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:16.148445 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 358/733] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:14:16.149550 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbfcca3-e4b4-495d-82b4-ce57dc72a658 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:16.164658 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 376/734] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:14:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:16.170732 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19ada83-e8c6-4988-a0f1-42b2b8cd8d4f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:16.247014 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:16.281462 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19ada83-e8c6-4988-a0f1-42b2b8cd8d4f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:16.293901 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbfcca3-e4b4-495d-82b4-ce57dc72a658 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:16.313333 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:16.318825 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d22c7-4dff-4366-964c-3bb0e410ddea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:16.320912 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d22c7-4dff-4366-964c-3bb0e410ddea None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:16.321397 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d22c7-4dff-4366-964c-3bb0e410ddea None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:16.321887 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d22c7-4dff-4366-964c-3bb0e410ddea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:16.337041 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e52811-d918-4aca-bc58-87ddf2c38e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:16.337640 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e52811-d918-4aca-bc58-87ddf2c38e80 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:16.337917 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e52811-d918-4aca-bc58-87ddf2c38e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:16.338187 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e52811-d918-4aca-bc58-87ddf2c38e80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:16.338765 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4d22c7-4dff-4366-964c-3bb0e410ddea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:16.340953 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4d22c7-4dff-4366-964c-3bb0e410ddea None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:16.349056 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e52811-d918-4aca-bc58-87ddf2c38e80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:16.350556 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e52811-d918-4aca-bc58-87ddf2c38e80 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:16.360797 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-4b4d22c7-4dff-4366-964c-3bb0e410ddea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:16.360797 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:16.360797 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:16.360797 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:16.360797 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:16.360797 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:16.360797 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:16.360797 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:16.360797 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:16.360797 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:16.360797 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:16.360797 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:16.360797 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:16.360797 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:16.360797 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:16.361752 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:16.361752 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:16.361752 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:16.361752 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:16.361752 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:16.361752 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:16.361752 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:16.361752 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:16.361752 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:16.361752 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:16.361752 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:16.361752 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:16.361752 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:16.361752 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:16.361752 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:16.361752 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:16.361752 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:16.362766 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:16.362766 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:16.362766 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:16.362766 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:16.368643 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-44e52811-d918-4aca-bc58-87ddf2c38e80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:16.368643 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:16.368643 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:16.368643 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:16.368643 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:16.368643 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:16.368643 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:16.368643 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:16.368643 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:16.368643 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:16.368643 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:16.368643 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:16.368643 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:16.368643 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:16.368643 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:16.369511 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:16.369511 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:16.369511 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:16.369511 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:16.369511 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:16.369511 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:16.369511 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:16.369511 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:16.369511 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:16.369511 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:16.369511 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:16.369511 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:16.369511 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:16.369511 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:16.369511 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:16.369511 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:16.369511 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:16.370402 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:16.370402 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:16.370402 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:16.370402 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:16.370614 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 377/735] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:14:16.370614 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d22c7-4dff-4366-964c-3bb0e410ddea None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:16.372575 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 359/736] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:16.373503 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e52811-d918-4aca-bc58-87ddf2c38e80 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:16.489567 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e52811-d918-4aca-bc58-87ddf2c38e80 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:16.496572 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d22c7-4dff-4366-964c-3bb0e410ddea None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:16.513774 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:16.534653 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4c6c64-afaf-4831-9ef7-215854d30a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:16.535002 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc040bc-d07f-41bd-bd56-62533a2188a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:16.535635 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c6c64-afaf-4831-9ef7-215854d30a75 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:16.535957 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc040bc-d07f-41bd-bd56-62533a2188a6 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:16.536082 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c6c64-afaf-4831-9ef7-215854d30a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:16.536440 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c6c64-afaf-4831-9ef7-215854d30a75 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:16.536872 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc040bc-d07f-41bd-bd56-62533a2188a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:16.537278 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc040bc-d07f-41bd-bd56-62533a2188a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:16.537386 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4c6c64-afaf-4831-9ef7-215854d30a75 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:16.538207 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc040bc-d07f-41bd-bd56-62533a2188a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:16.538647 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc040bc-d07f-41bd-bd56-62533a2188a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:16.539050 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4c6c64-afaf-4831-9ef7-215854d30a75 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:16.540360 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc040bc-d07f-41bd-bd56-62533a2188a6 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:16.555417 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 360/737] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:16.556434 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc040bc-d07f-41bd-bd56-62533a2188a6 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:16.566184 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 378/738] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 5 10:14:16 2022] POST /identity/v3/services => generated 251 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:14:16.572490 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4c6c64-afaf-4831-9ef7-215854d30a75 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:16.670305 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc040bc-d07f-41bd-bd56-62533a2188a6 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:16.687136 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:16.734227 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90641b0f-e6d8-4ae0-aa8a-7adb197a2bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:16.735202 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90641b0f-e6d8-4ae0-aa8a-7adb197a2bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:16.735628 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90641b0f-e6d8-4ae0-aa8a-7adb197a2bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:16.736046 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90641b0f-e6d8-4ae0-aa8a-7adb197a2bd4 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:16.737200 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90641b0f-e6d8-4ae0-aa8a-7adb197a2bd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:16.737672 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90641b0f-e6d8-4ae0-aa8a-7adb197a2bd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:16.739266 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90641b0f-e6d8-4ae0-aa8a-7adb197a2bd4 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:16.753479 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 361/739] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:16.764041 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90641b0f-e6d8-4ae0-aa8a-7adb197a2bd4 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:16.783002 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4c6c64-afaf-4831-9ef7-215854d30a75 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:16.818976 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:16.827372 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d6aee-bd30-4e0c-ab76-a89cdd61a5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:16.828035 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d6aee-bd30-4e0c-ab76-a89cdd61a5f2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:16.828311 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d6aee-bd30-4e0c-ab76-a89cdd61a5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:16.828580 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d6aee-bd30-4e0c-ab76-a89cdd61a5f2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:16.842224 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952d6aee-bd30-4e0c-ab76-a89cdd61a5f2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:16.844015 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952d6aee-bd30-4e0c-ab76-a89cdd61a5f2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:16.858189 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-952d6aee-bd30-4e0c-ab76-a89cdd61a5f2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 05 10:14:16.858189 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:16.858189 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:16.858189 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:16.858189 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:16.858189 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:16.858189 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:16.858189 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:16.858189 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:16.858189 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:16.858189 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:16.858189 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:16.858189 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 10:14:16.858189 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 10:14:16.858189 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 10:14:16.858189 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 10:14:16.859213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:16.859213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:16.859213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 10:14:16.859213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:16.859213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 10:14:16.859213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:16.859213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:16.859213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:16.859213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:16.859213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:16.859213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:16.859213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:16.859213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:16.859213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:16.859213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:16.859213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:16.859213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:16.859213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:16.860211 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:16.860211 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:16.860211 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 10:14:16.860211 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 10:14:16.860211 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 10:14:16.860211 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 10:14:16.860211 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 10:14:16.860211 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 10:14:16.860211 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 05 10:14:16.860211 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:16.860211 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 379/740] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 10:14:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:16.865646 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d6aee-bd30-4e0c-ab76-a89cdd61a5f2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:16.902480 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90641b0f-e6d8-4ae0-aa8a-7adb197a2bd4 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:16.943183 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285d2348-0e9d-4844-9406-74cdc24e6c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:16.944338 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d2348-0e9d-4844-9406-74cdc24e6c8d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:16.944826 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d2348-0e9d-4844-9406-74cdc24e6c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:16.945322 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d2348-0e9d-4844-9406-74cdc24e6c8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:16.954558 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285d2348-0e9d-4844-9406-74cdc24e6c8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:16.956517 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285d2348-0e9d-4844-9406-74cdc24e6c8d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:16.970444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-285d2348-0e9d-4844-9406-74cdc24e6c8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:16.970444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:16.970444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:16.970444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:16.970444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:16.970444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:16.970444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:16.970444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:16.970444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:16.970444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:16.970444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:16.970444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:16.970444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 10:14:16.970444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 10:14:16.970444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 10:14:16.971701 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 10:14:16.971701 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:16.971701 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:16.971701 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:16.971701 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:16.971701 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:16.971701 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:16.971701 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:16.971701 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:16.971701 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:16.971701 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:16.971701 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:16.971701 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:16.971701 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:16.971701 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:16.971701 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:16.971701 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:16.972642 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 10:14:16.972642 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 10:14:16.972642 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 10:14:16.972642 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 10:14:16.972642 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 10:14:16.972642 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 10:14:16.972642 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:16.972642 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:16.975968 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 362/741] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:14:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:16.982599 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285d2348-0e9d-4844-9406-74cdc24e6c8d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:16.986259 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d6aee-bd30-4e0c-ab76-a89cdd61a5f2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:16.989149 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:17.025521 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c7e5f1-ec4d-4056-8543-2bf92a539606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:17.026072 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c7e5f1-ec4d-4056-8543-2bf92a539606 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:17.026355 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c7e5f1-ec4d-4056-8543-2bf92a539606 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:17.026691 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c7e5f1-ec4d-4056-8543-2bf92a539606 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:17.027895 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c7e5f1-ec4d-4056-8543-2bf92a539606 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:17.028298 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c7e5f1-ec4d-4056-8543-2bf92a539606 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:17.029894 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c7e5f1-ec4d-4056-8543-2bf92a539606 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:17.050520 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 380/742] 10.222.0.40 () {56 vars in 1135 bytes} [Fri Aug 5 10:14:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:17.055788 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c7e5f1-ec4d-4056-8543-2bf92a539606 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:17.120688 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:17.132107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285d2348-0e9d-4844-9406-74cdc24e6c8d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:17.162744 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c7e5f1-ec4d-4056-8543-2bf92a539606 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:17.204246 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f0443-7284-4c0d-9201-5b5eb99c9cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:17.205110 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f0443-7284-4c0d-9201-5b5eb99c9cbf None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:17.205502 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f0443-7284-4c0d-9201-5b5eb99c9cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:17.205883 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f0443-7284-4c0d-9201-5b5eb99c9cbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:17.209468 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b04a2b-c8f2-4ca3-81f4-ea52d82c943a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:17.210106 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b04a2b-c8f2-4ca3-81f4-ea52d82c943a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:17.210391 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b04a2b-c8f2-4ca3-81f4-ea52d82c943a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:17.210663 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b04a2b-c8f2-4ca3-81f4-ea52d82c943a None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:17.211508 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b04a2b-c8f2-4ca3-81f4-ea52d82c943a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:17.211816 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b04a2b-c8f2-4ca3-81f4-ea52d82c943a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:17.213256 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b04a2b-c8f2-4ca3-81f4-ea52d82c943a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:17.234717 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 363/743] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 10:14:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:17.239887 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:17.241392 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b04a2b-c8f2-4ca3-81f4-ea52d82c943a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:17.279686 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565f0443-7284-4c0d-9201-5b5eb99c9cbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:17.281103 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565f0443-7284-4c0d-9201-5b5eb99c9cbf None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:17.397502 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b04a2b-c8f2-4ca3-81f4-ea52d82c943a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:17.420028 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 381/744] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:14:17 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 10:14:17.421459 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d82346-a9f4-4044-a220-1faad404439b None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:17.421745 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d82346-a9f4-4044-a220-1faad404439b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:17.422021 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d82346-a9f4-4044-a220-1faad404439b None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:17.427583 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 382/745] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:17.428358 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d82346-a9f4-4044-a220-1faad404439b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:17.445358 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e49b4b-b62d-4ca6-89f5-85d45924ed25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:17.446244 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e49b4b-b62d-4ca6-89f5-85d45924ed25 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:17.446520 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e49b4b-b62d-4ca6-89f5-85d45924ed25 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:17.446787 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e49b4b-b62d-4ca6-89f5-85d45924ed25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:17.447895 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e49b4b-b62d-4ca6-89f5-85d45924ed25 None admin] RBAC: Adding query filter params (role.id=3a2b4319820a488198dabee34420cdc7, scope.project.id=36b8b505012d4d0bac54845b6799e743, user.id=52fc3ba90dc045c2a5b7bb730da96acc) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:17.448576 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e49b4b-b62d-4ca6-89f5-85d45924ed25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:17.450473 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e49b4b-b62d-4ca6-89f5-85d45924ed25 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:17.458429 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 364/746] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:14:17 2022] GET /identity/v3/role_assignments?user.id=52fc3ba90dc045c2a5b7bb730da96acc&scope.project.id=36b8b505012d4d0bac54845b6799e743&role.id=3a2b4319820a488198dabee34420cdc7 => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:14:17.460222 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c78705-c294-4d7e-bcf8-814e5eafb75a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:17.460507 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c78705-c294-4d7e-bcf8-814e5eafb75a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:17.460837 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c78705-c294-4d7e-bcf8-814e5eafb75a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:17.493698 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-66c78705-c294-4d7e-bcf8-814e5eafb75a None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:17.494960 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-66c78705-c294-4d7e-bcf8-814e5eafb75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:17.525015 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:17.533566 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:17.564304 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d82346-a9f4-4044-a220-1faad404439b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:17.626394 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c7be4c-1fbd-4f34-969a-058209e3e91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:17.627088 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7be4c-1fbd-4f34-969a-058209e3e91d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:17.627371 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7be4c-1fbd-4f34-969a-058209e3e91d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:17.627652 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7be4c-1fbd-4f34-969a-058209e3e91d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:17.629703 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 365/747] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:17 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:14:17.631329 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df55532-8c34-49e9-92f9-a2bc6a9ca423 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:17.631615 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df55532-8c34-49e9-92f9-a2bc6a9ca423 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:17.631886 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df55532-8c34-49e9-92f9-a2bc6a9ca423 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:17.638299 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 366/748] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:17.639720 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c06721-d32e-411c-a362-e8d1253ec53a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:17.640008 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c06721-d32e-411c-a362-e8d1253ec53a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:17.640284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c06721-d32e-411c-a362-e8d1253ec53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:17.652974 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c7be4c-1fbd-4f34-969a-058209e3e91d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:17.654508 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c7be4c-1fbd-4f34-969a-058209e3e91d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:17.666018 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b4c06721-d32e-411c-a362-e8d1253ec53a None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:17.667254 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b4c06721-d32e-411c-a362-e8d1253ec53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:17.675323 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-32c7be4c-1fbd-4f34-969a-058209e3e91d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:17.675323 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:17.675323 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:17.675323 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:17.675323 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:17.675323 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:17.675323 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:17.675323 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:17.675323 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:17.675323 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:17.675323 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:17.675323 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:17.675323 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:17.675323 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:17.675323 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:17.676268 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:17.676268 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:17.676268 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:17.676268 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:17.676268 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:17.676268 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:17.676268 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:17.676268 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:17.676268 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:17.676268 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:17.676268 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:17.676268 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:17.676268 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:17.676268 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:17.676268 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:17.676268 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:17.676268 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:17.677165 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:17.677165 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:17.677165 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:17.677165 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:17.679385 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 383/749] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:17.680393 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c7be4c-1fbd-4f34-969a-058209e3e91d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:17.745077 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 367/750] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:17 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 10:14:17.750938 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c06721-d32e-411c-a362-e8d1253ec53a None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:17.824867 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c7be4c-1fbd-4f34-969a-058209e3e91d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:17.839882 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:17.854304 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:17.863393 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa238a9-3221-4856-aac5-ea0a5eb20011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:17.864310 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa238a9-3221-4856-aac5-ea0a5eb20011 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:17.864793 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa238a9-3221-4856-aac5-ea0a5eb20011 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:17.865261 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa238a9-3221-4856-aac5-ea0a5eb20011 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:17.866149 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa238a9-3221-4856-aac5-ea0a5eb20011 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:17.867946 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa238a9-3221-4856-aac5-ea0a5eb20011 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:17.877810 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c06721-d32e-411c-a362-e8d1253ec53a None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:17.887617 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 384/751] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 5 10:14:17 2022] POST /identity/v3/services => generated 240 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:14:17.889212 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa238a9-3221-4856-aac5-ea0a5eb20011 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:17.948754 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d38423-bd35-4fa5-933a-633d0039d4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:17.949502 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d38423-bd35-4fa5-933a-633d0039d4da None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:17.949781 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d38423-bd35-4fa5-933a-633d0039d4da None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:17.950050 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d38423-bd35-4fa5-933a-633d0039d4da None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:17.950774 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d38423-bd35-4fa5-933a-633d0039d4da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:17.951104 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d38423-bd35-4fa5-933a-633d0039d4da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:17.952543 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d38423-bd35-4fa5-933a-633d0039d4da None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:17.984243 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 368/752] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:17.990337 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d38423-bd35-4fa5-933a-633d0039d4da None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:18.102793 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa238a9-3221-4856-aac5-ea0a5eb20011 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:18.146118 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ba9ce7-96f7-4a2e-be87-763e4d00da3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:18.147191 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba9ce7-96f7-4a2e-be87-763e4d00da3f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:18.147682 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba9ce7-96f7-4a2e-be87-763e4d00da3f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:18.148163 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba9ce7-96f7-4a2e-be87-763e4d00da3f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:18.149119 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d38423-bd35-4fa5-933a-633d0039d4da None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:18.165070 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ba9ce7-96f7-4a2e-be87-763e4d00da3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:14:18.171652 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-f3ba9ce7-96f7-4a2e-be87-763e4d00da3f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:14:18.171652 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:18.171652 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:18.171652 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:18.171652 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:18.171652 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:18.171652 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:18.171652 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:18.171652 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:18.171652 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:18.171652 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:18.171652 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:18.171652 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:14:18.171652 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:14:18.171652 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:14:18.171652 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:14:18.172581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:14:18.172581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:14:18.172581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:14:18.172581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:14:18.172581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:18.172581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:18.172581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:18.172581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:18.172581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:18.172581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:18.172581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:18.172581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:18.172581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:18.172581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:18.172581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:18.172581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:18.173445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:18.173445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:18.173445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:18.173445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:18.173445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:14:18.173445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:14:18.173445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:14:18.173445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:14:18.173445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:14:18.173445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:14:18.173445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:14:18.173445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:18.176144 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 385/753] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 10:14:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:18.184920 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ba9ce7-96f7-4a2e-be87-763e4d00da3f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:18.208013 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cee1208-751c-419c-b498-2d022504a094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:18.208721 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee1208-751c-419c-b498-2d022504a094 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:18.209006 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee1208-751c-419c-b498-2d022504a094 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:18.209277 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee1208-751c-419c-b498-2d022504a094 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:18.212593 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:18.218772 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cee1208-751c-419c-b498-2d022504a094 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:18.221292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cee1208-751c-419c-b498-2d022504a094 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:18.229192 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:18.238474 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-4cee1208-751c-419c-b498-2d022504a094 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:18.238474 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:18.238474 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:18.238474 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:18.238474 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:18.238474 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:18.238474 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:18.238474 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:18.238474 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:18.238474 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:18.238474 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:18.238474 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:18.238474 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 10:14:18.238474 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 10:14:18.238474 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 10:14:18.239877 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 10:14:18.239877 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:18.239877 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:18.239877 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:18.239877 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:18.239877 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:18.239877 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:18.239877 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:18.239877 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:18.239877 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:18.239877 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:18.239877 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:18.239877 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:18.239877 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:18.239877 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:18.239877 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:18.239877 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:18.241082 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 10:14:18.241082 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 10:14:18.241082 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 10:14:18.241082 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 10:14:18.241082 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 10:14:18.241082 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 10:14:18.241082 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:18.241082 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:18.241512 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 369/754] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:14:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:14:18.247632 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cee1208-751c-419c-b498-2d022504a094 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:18.317971 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ba9ce7-96f7-4a2e-be87-763e4d00da3f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:18.354342 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09266f1-72cf-439e-9a5b-f6ce70000b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:18.355032 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09266f1-72cf-439e-9a5b-f6ce70000b44 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:18.355361 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09266f1-72cf-439e-9a5b-f6ce70000b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:18.355815 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09266f1-72cf-439e-9a5b-f6ce70000b44 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:18.356786 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09266f1-72cf-439e-9a5b-f6ce70000b44 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:18.357148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09266f1-72cf-439e-9a5b-f6ce70000b44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:18.358666 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09266f1-72cf-439e-9a5b-f6ce70000b44 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:18.367857 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 386/755] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:14:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:14:18.378659 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09266f1-72cf-439e-9a5b-f6ce70000b44 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:18.414205 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cee1208-751c-419c-b498-2d022504a094 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:18.449828 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909a716-2626-46f9-8b61-b60066ec2f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:18.450236 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909a716-2626-46f9-8b61-b60066ec2f45 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:18.450527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909a716-2626-46f9-8b61-b60066ec2f45 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:18.450800 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909a716-2626-46f9-8b61-b60066ec2f45 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:18.451590 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3909a716-2626-46f9-8b61-b60066ec2f45 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:18.451903 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3909a716-2626-46f9-8b61-b60066ec2f45 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:18.453351 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3909a716-2626-46f9-8b61-b60066ec2f45 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:18.466410 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 370/756] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 10:14:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:18.479847 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909a716-2626-46f9-8b61-b60066ec2f45 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:18.494223 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:18.511536 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09266f1-72cf-439e-9a5b-f6ce70000b44 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:18.575281 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:18.579164 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cc343a-a5c4-43ae-8391-9d588788b2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:18.579880 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc343a-a5c4-43ae-8391-9d588788b2da None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:18.580170 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc343a-a5c4-43ae-8391-9d588788b2da None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:18.580443 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc343a-a5c4-43ae-8391-9d588788b2da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:18.600479 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cc343a-a5c4-43ae-8391-9d588788b2da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:18.603530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cc343a-a5c4-43ae-8391-9d588788b2da None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:18.633172 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-52cc343a-a5c4-43ae-8391-9d588788b2da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:18.633172 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:18.633172 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:18.633172 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:18.633172 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:18.633172 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:18.633172 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:18.633172 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:18.633172 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:18.633172 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:18.633172 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:18.633172 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:18.633172 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:18.633172 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:18.633172 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:18.635389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:18.635389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:18.635389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:18.635389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:18.635389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:18.635389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:18.635389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:18.635389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:18.635389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:18.635389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:18.635389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:18.635389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:18.635389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:18.635389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:18.635389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:18.635389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:18.635389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:18.638280 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:18.638280 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:18.638280 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:18.638280 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:18.642363 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 387/757] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:14:18.644736 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cc343a-a5c4-43ae-8391-9d588788b2da None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:18.660885 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909a716-2626-46f9-8b61-b60066ec2f45 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:18.710664 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bdaf97-ab4f-46b9-b37c-9087ffb66ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:18.711352 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bdaf97-ab4f-46b9-b37c-9087ffb66ef7 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:18.711634 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bdaf97-ab4f-46b9-b37c-9087ffb66ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:18.711901 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bdaf97-ab4f-46b9-b37c-9087ffb66ef7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:18.712708 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bdaf97-ab4f-46b9-b37c-9087ffb66ef7 None admin] RBAC: Adding query filter params (role.id=daef89adcaeb48a382a0bcb03340ccbb, scope.project.id=36b8b505012d4d0bac54845b6799e743, user.id=c80cf0e429f04f229aea1aa9293a93b2) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:18.713014 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bdaf97-ab4f-46b9-b37c-9087ffb66ef7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:18.714467 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bdaf97-ab4f-46b9-b37c-9087ffb66ef7 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:18.728709 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 371/758] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:14:18 2022] GET /identity/v3/role_assignments?user.id=c80cf0e429f04f229aea1aa9293a93b2&scope.project.id=36b8b505012d4d0bac54845b6799e743&role.id=daef89adcaeb48a382a0bcb03340ccbb => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:18.803769 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cc343a-a5c4-43ae-8391-9d588788b2da None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:18.861924 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd23f758-dac1-49aa-b777-b626b01ccd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:18.862608 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd23f758-dac1-49aa-b777-b626b01ccd66 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:18.862890 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd23f758-dac1-49aa-b777-b626b01ccd66 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:18.863220 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd23f758-dac1-49aa-b777-b626b01ccd66 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:18.863953 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd23f758-dac1-49aa-b777-b626b01ccd66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:18.864437 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd23f758-dac1-49aa-b777-b626b01ccd66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:18.865909 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd23f758-dac1-49aa-b777-b626b01ccd66 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:18.885464 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 388/759] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:14:18.892565 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bdaf97-ab4f-46b9-b37c-9087ffb66ef7 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:19.034248 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bdaf97-ab4f-46b9-b37c-9087ffb66ef7 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:19.059988 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:19.079106 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b53292-5ed4-4367-8bc6-f093e75e4890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:19.079764 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b53292-5ed4-4367-8bc6-f093e75e4890 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:19.080045 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b53292-5ed4-4367-8bc6-f093e75e4890 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:19.080317 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b53292-5ed4-4367-8bc6-f093e75e4890 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:19.095982 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b53292-5ed4-4367-8bc6-f093e75e4890 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:19.097456 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b53292-5ed4-4367-8bc6-f093e75e4890 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:19.112465 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-d5b53292-5ed4-4367-8bc6-f093e75e4890 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 10:14:19.112465 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:19.112465 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:19.112465 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:19.112465 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:19.112465 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:19.112465 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:19.112465 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:19.112465 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:19.112465 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:19.112465 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:19.112465 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:19.112465 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 10:14:19.112465 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 10:14:19.112465 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 10:14:19.112465 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 10:14:19.113452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:19.113452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:19.113452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 10:14:19.113452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:19.113452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 10:14:19.113452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:19.113452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:19.113452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:19.113452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:19.113452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:19.113452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:19.113452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:19.113452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:19.113452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:19.113452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:19.113452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:19.113452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:19.113452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:19.114373 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:19.114373 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:19.114373 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 10:14:19.114373 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 10:14:19.114373 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 10:14:19.114373 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 10:14:19.114373 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 10:14:19.114373 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 10:14:19.114373 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 10:14:19.114373 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:19.114373 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 372/760] 10.222.0.40 () {56 vars in 1093 bytes} [Fri Aug 5 10:14:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:19.119580 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd23f758-dac1-49aa-b777-b626b01ccd66 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:19.133189 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:19.177011 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926fa2f7-f094-4fe3-9855-a6847cd4586f None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:19.177608 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926fa2f7-f094-4fe3-9855-a6847cd4586f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:19.178089 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926fa2f7-f094-4fe3-9855-a6847cd4586f None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:19.180096 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 373/761] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 10:14:19.186159 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a221cd71-8ffc-47a2-b80e-65e2dd3e48a6 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:19.186749 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a221cd71-8ffc-47a2-b80e-65e2dd3e48a6 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:19.187288 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a221cd71-8ffc-47a2-b80e-65e2dd3e48a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:19.208216 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a221cd71-8ffc-47a2-b80e-65e2dd3e48a6 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:19.209790 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a221cd71-8ffc-47a2-b80e-65e2dd3e48a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:19.314848 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd23f758-dac1-49aa-b777-b626b01ccd66 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:19.326425 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 374/762] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 10:14:19.348153 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a221cd71-8ffc-47a2-b80e-65e2dd3e48a6 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:19.388516 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95325c40-ebf9-4fb1-b52b-afcd8096c848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:19.389157 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95325c40-ebf9-4fb1-b52b-afcd8096c848 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:19.389434 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95325c40-ebf9-4fb1-b52b-afcd8096c848 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:19.389702 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95325c40-ebf9-4fb1-b52b-afcd8096c848 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:19.390518 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95325c40-ebf9-4fb1-b52b-afcd8096c848 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:19.390827 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95325c40-ebf9-4fb1-b52b-afcd8096c848 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:19.393279 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95325c40-ebf9-4fb1-b52b-afcd8096c848 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:19.430247 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 389/763] 10.222.0.40 () {56 vars in 1133 bytes} [Fri Aug 5 10:14:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:19.439920 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95325c40-ebf9-4fb1-b52b-afcd8096c848 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:19.508060 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:19.555674 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95325c40-ebf9-4fb1-b52b-afcd8096c848 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:19.568651 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a221cd71-8ffc-47a2-b80e-65e2dd3e48a6 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:19.588964 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:19.622733 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b223045-1e23-4b48-a089-5a0d71e082bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:19.623436 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b223045-1e23-4b48-a089-5a0d71e082bc None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:19.623687 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b223045-1e23-4b48-a089-5a0d71e082bc None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:19.623961 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b223045-1e23-4b48-a089-5a0d71e082bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:19.636018 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5014a74-4099-4c13-9e7d-984bc9f721b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:19.636755 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5014a74-4099-4c13-9e7d-984bc9f721b7 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:19.637033 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5014a74-4099-4c13-9e7d-984bc9f721b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:19.637306 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5014a74-4099-4c13-9e7d-984bc9f721b7 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:19.637946 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5014a74-4099-4c13-9e7d-984bc9f721b7 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:19.639358 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5014a74-4099-4c13-9e7d-984bc9f721b7 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:19.645687 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b223045-1e23-4b48-a089-5a0d71e082bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:19.647958 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b223045-1e23-4b48-a089-5a0d71e082bc None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:19.660833 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-c5014a74-4099-4c13-9e7d-984bc9f721b7 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 10:14:19.660833 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:19.660833 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 10:14:19.660833 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 10:14:19.660833 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 10:14:19.660833 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 10:14:19.660833 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 10:14:19.660833 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:19.660833 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 10:14:19.660833 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:19.660833 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 10:14:19.660833 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:19.660833 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:19.660833 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:19.660833 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:19.660833 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:19.661769 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:19.661769 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:19.661769 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:19.661769 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:19.661769 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:19.661769 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:19.661769 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:19.661769 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 10:14:19.661769 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 10:14:19.661769 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 10:14:19.661769 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 10:14:19.661769 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:19.661769 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:19.661769 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:19.661769 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:19.661769 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:19.662609 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:19.662609 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:19.662609 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:19.662609 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:19.662609 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:19.662609 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:19.662609 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:19.662609 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:19.662609 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:19.662609 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:19.662609 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:19.662609 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 10:14:19.662609 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:19.662609 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 10:14:19.662609 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:19.662609 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 375/764] 10.222.0.40 () {56 vars in 1129 bytes} [Fri Aug 5 10:14:19 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:14:19.671429 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5014a74-4099-4c13-9e7d-984bc9f721b7 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:19.681548 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-8b223045-1e23-4b48-a089-5a0d71e082bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:19.681548 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:19.681548 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:19.681548 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:19.681548 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:19.681548 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:19.681548 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:19.681548 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:19.681548 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:19.681548 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:19.681548 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:19.681548 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:19.681548 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:19.681548 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:19.681548 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:19.682446 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:19.682446 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:19.682446 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:19.682446 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:19.682446 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:19.682446 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:19.682446 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:19.682446 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:19.682446 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:19.682446 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:19.682446 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:19.682446 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:19.682446 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:19.682446 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:19.682446 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:19.682446 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:19.682446 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:19.683413 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:19.683413 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:19.683413 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:19.683413 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:19.685842 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 390/765] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:19.690133 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b223045-1e23-4b48-a089-5a0d71e082bc None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:19.792593 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b223045-1e23-4b48-a089-5a0d71e082bc None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:19.810546 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:19.829070 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e707ecc-5783-4ce6-8bd5-3f83620cf785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:19.830858 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e707ecc-5783-4ce6-8bd5-3f83620cf785 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:19.831166 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e707ecc-5783-4ce6-8bd5-3f83620cf785 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:19.831439 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e707ecc-5783-4ce6-8bd5-3f83620cf785 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:19.832139 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e707ecc-5783-4ce6-8bd5-3f83620cf785 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:19.832441 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e707ecc-5783-4ce6-8bd5-3f83620cf785 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:19.833817 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e707ecc-5783-4ce6-8bd5-3f83620cf785 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:19.860076 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 391/766] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:14:19.865740 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e707ecc-5783-4ce6-8bd5-3f83620cf785 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:19.897940 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5014a74-4099-4c13-9e7d-984bc9f721b7 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:19.940935 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af77b97-6c73-4649-af70-0694309b1140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:19.942074 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af77b97-6c73-4649-af70-0694309b1140 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:19.942404 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af77b97-6c73-4649-af70-0694309b1140 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:19.942966 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af77b97-6c73-4649-af70-0694309b1140 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:19.943665 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af77b97-6c73-4649-af70-0694309b1140 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:19.943966 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af77b97-6c73-4649-af70-0694309b1140 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:19.946526 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af77b97-6c73-4649-af70-0694309b1140 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:19.952203 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 376/767] 10.222.0.40 () {56 vars in 1124 bytes} [Fri Aug 5 10:14:19 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:19.954473 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e0e01-1c43-4742-9794-daffdb80dab9 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:19.954759 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e0e01-1c43-4742-9794-daffdb80dab9 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:19.955196 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e0e01-1c43-4742-9794-daffdb80dab9 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:19.956958 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 377/768] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:19.957960 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84e0e01-1c43-4742-9794-daffdb80dab9 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:19.979264 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e707ecc-5783-4ce6-8bd5-3f83620cf785 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:20.012464 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:20.032860 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b1e9c-d93d-475f-b90b-a19efe383326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:20.033526 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b1e9c-d93d-475f-b90b-a19efe383326 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:20.034051 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b1e9c-d93d-475f-b90b-a19efe383326 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:20.034330 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b1e9c-d93d-475f-b90b-a19efe383326 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:20.045960 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601b1e9c-d93d-475f-b90b-a19efe383326 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:20.048283 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601b1e9c-d93d-475f-b90b-a19efe383326 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:20.060862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-601b1e9c-d93d-475f-b90b-a19efe383326 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:20.060862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:20.060862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:20.060862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:20.060862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:20.060862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:20.060862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:20.060862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:20.060862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:20.060862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:20.060862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:20.060862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:20.060862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 10:14:20.060862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 10:14:20.060862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 10:14:20.061848 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 10:14:20.061848 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:20.061848 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:20.061848 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:20.061848 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:20.061848 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:20.061848 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:20.061848 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:20.061848 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:20.061848 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:20.061848 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:20.061848 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:20.061848 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:20.061848 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:20.061848 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:20.061848 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:20.061848 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:20.062718 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 10:14:20.062718 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 10:14:20.062718 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 10:14:20.062718 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 10:14:20.062718 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 10:14:20.062718 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 10:14:20.062718 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:20.062718 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:20.067144 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 392/769] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:14:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:14:20.068713 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bcdf11-6b64-4fd8-a338-bddffc2841e5 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:20.069013 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bcdf11-6b64-4fd8-a338-bddffc2841e5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:20.069291 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bcdf11-6b64-4fd8-a338-bddffc2841e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:20.086511 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84e0e01-1c43-4742-9794-daffdb80dab9 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:20.088551 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b5bcdf11-6b64-4fd8-a338-bddffc2841e5 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:20.089883 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b5bcdf11-6b64-4fd8-a338-bddffc2841e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:20.122187 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:20.133718 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30f887-3ea4-4c7a-941b-b7911cd73cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:20.134362 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30f887-3ea4-4c7a-941b-b7911cd73cdf None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:20.134643 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30f887-3ea4-4c7a-941b-b7911cd73cdf None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:20.134937 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30f887-3ea4-4c7a-941b-b7911cd73cdf None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:20.135643 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b30f887-3ea4-4c7a-941b-b7911cd73cdf None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:20.135950 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b30f887-3ea4-4c7a-941b-b7911cd73cdf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:20.137550 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b30f887-3ea4-4c7a-941b-b7911cd73cdf None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:20.143540 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 378/770] 10.222.0.40 () {56 vars in 1124 bytes} [Fri Aug 5 10:14:19 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:20.144447 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30f887-3ea4-4c7a-941b-b7911cd73cdf None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:20.180114 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 393/771] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:14:20.181627 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bcdf11-6b64-4fd8-a338-bddffc2841e5 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:20.276563 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30f887-3ea4-4c7a-941b-b7911cd73cdf None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:20.279802 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:20.295517 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bcdf11-6b64-4fd8-a338-bddffc2841e5 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:20.324632 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2993601-fe6d-48ac-b217-45484d883fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:20.325841 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2993601-fe6d-48ac-b217-45484d883fee None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:20.326551 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2993601-fe6d-48ac-b217-45484d883fee None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:20.327398 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2993601-fe6d-48ac-b217-45484d883fee None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:20.328611 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2993601-fe6d-48ac-b217-45484d883fee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:20.329257 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2993601-fe6d-48ac-b217-45484d883fee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:20.331273 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2993601-fe6d-48ac-b217-45484d883fee None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:20.332137 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b34b120-ce83-49a7-b750-2d91ee7747c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:20.332769 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34b120-ce83-49a7-b750-2d91ee7747c8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:20.333102 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34b120-ce83-49a7-b750-2d91ee7747c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:20.333379 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34b120-ce83-49a7-b750-2d91ee7747c8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:20.334243 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b34b120-ce83-49a7-b750-2d91ee7747c8 None admin] RBAC: Adding query filter params (interface=public, service_id=ec3754794f934efcb413749f36c55a83, region_id=RegionOne) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:20.334556 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b34b120-ce83-49a7-b750-2d91ee7747c8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:20.336068 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b34b120-ce83-49a7-b750-2d91ee7747c8 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:20.344559 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 394/772] 10.222.0.40 () {56 vars in 1251 bytes} [Fri Aug 5 10:14:20 2022] GET /identity/v3/endpoints?service_id=ec3754794f934efcb413749f36c55a83&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:20.345482 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b34b120-ce83-49a7-b750-2d91ee7747c8 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:20.359520 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 379/773] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 10:14:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:20.360742 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2993601-fe6d-48ac-b217-45484d883fee None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:20.390290 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:20.467329 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2993601-fe6d-48ac-b217-45484d883fee None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:20.471622 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b34b120-ce83-49a7-b750-2d91ee7747c8 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:20.506211 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebbc88f-70c7-4d0b-9f46-77a827ed9b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:20.506835 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbc88f-70c7-4d0b-9f46-77a827ed9b9e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:20.507041 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d247915-cf60-469e-8541-0df4c0022b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:20.507172 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbc88f-70c7-4d0b-9f46-77a827ed9b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:20.507398 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbc88f-70c7-4d0b-9f46-77a827ed9b9e None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:20.507670 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d247915-cf60-469e-8541-0df4c0022b2b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:20.507949 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d247915-cf60-469e-8541-0df4c0022b2b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:20.508045 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebbc88f-70c7-4d0b-9f46-77a827ed9b9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:20.508221 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d247915-cf60-469e-8541-0df4c0022b2b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:20.509392 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebbc88f-70c7-4d0b-9f46-77a827ed9b9e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:20.515200 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 380/774] 10.222.0.40 () {56 vars in 1087 bytes} [Fri Aug 5 10:14:20 2022] GET /identity/v3/services => generated 772 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:20.519965 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d247915-cf60-469e-8541-0df4c0022b2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:14:20.520486 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebbc88f-70c7-4d0b-9f46-77a827ed9b9e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:20.525923 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-9d247915-cf60-469e-8541-0df4c0022b2b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 10:14:20.525923 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:20.525923 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:20.525923 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:20.525923 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:20.525923 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:20.525923 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:20.525923 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:20.525923 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:20.525923 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:20.525923 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:20.525923 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:20.525923 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:14:20.525923 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:14:20.525923 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:14:20.525923 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:14:20.528362 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:14:20.528362 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:14:20.528362 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:14:20.528362 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:14:20.528362 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:20.528362 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:20.528362 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:20.528362 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:20.528362 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:20.528362 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:20.528362 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:20.528362 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:20.528362 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:20.528362 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:20.528362 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:20.528362 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:20.530090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:20.530090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:20.530090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:20.530090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:20.530090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:14:20.530090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:14:20.530090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:14:20.530090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:14:20.530090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:14:20.530090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:14:20.530090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 10:14:20.530090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:20.530090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 395/775] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 10:14:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:20.530090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787c084-48a3-4e3b-843d-d23096519e3b None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:20.533033 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787c084-48a3-4e3b-843d-d23096519e3b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:20.533033 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787c084-48a3-4e3b-843d-d23096519e3b None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:20.533033 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 396/776] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:20.533490 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c787c084-48a3-4e3b-843d-d23096519e3b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:20.567574 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:20.648656 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:20.663412 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c787c084-48a3-4e3b-843d-d23096519e3b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:20.669926 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebbc88f-70c7-4d0b-9f46-77a827ed9b9e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:20.714943 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d59eb-e64a-41c9-911c-7d8e4819f045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:20.715189 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb54cc4-da87-4a9b-8c68-fc747559907c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:20.716137 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb54cc4-da87-4a9b-8c68-fc747559907c None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:20.716137 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb54cc4-da87-4a9b-8c68-fc747559907c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:20.716317 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d59eb-e64a-41c9-911c-7d8e4819f045 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:20.716399 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb54cc4-da87-4a9b-8c68-fc747559907c None admin] PATH_INFO: `/v3/projects/36b8b505012d4d0bac54845b6799e743/users/b938f1f41db94b0eafe95bc8eeaae175/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:20.716472 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d59eb-e64a-41c9-911c-7d8e4819f045 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:20.716685 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d59eb-e64a-41c9-911c-7d8e4819f045 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:20.717574 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1d59eb-e64a-41c9-911c-7d8e4819f045 None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:20.718498 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1d59eb-e64a-41c9-911c-7d8e4819f045 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:20.718610 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb54cc4-da87-4a9b-8c68-fc747559907c None admin] RBAC: Authorizing `identity:create_grant(project_id=36b8b505012d4d0bac54845b6799e743, user_id=b938f1f41db94b0eafe95bc8eeaae175, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:20.719283 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1d59eb-e64a-41c9-911c-7d8e4819f045 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:20.720037 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb54cc4-da87-4a9b-8c68-fc747559907c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:20.728494 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 397/777] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 5 10:14:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:20.729932 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c771a-838b-4811-893a-e6796d675185 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:20.730401 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c771a-838b-4811-893a-e6796d675185 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:20.731371 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c771a-838b-4811-893a-e6796d675185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:20.742327 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 381/778] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 10:14:20 2022] PUT /identity/v3/projects/36b8b505012d4d0bac54845b6799e743/users/b938f1f41db94b0eafe95bc8eeaae175/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:14:20.743211 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb54cc4-da87-4a9b-8c68-fc747559907c None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:20.751913 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a30c771a-838b-4811-893a-e6796d675185 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:20.753378 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a30c771a-838b-4811-893a-e6796d675185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:20.844627 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 398/779] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:14:20.878180 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c771a-838b-4811-893a-e6796d675185 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:20.907753 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:20.925549 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb54cc4-da87-4a9b-8c68-fc747559907c None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:20.969263 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8c6948-1dcd-4555-bb6d-e85d58aad200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:20.970373 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c6948-1dcd-4555-bb6d-e85d58aad200 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:20.970857 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c6948-1dcd-4555-bb6d-e85d58aad200 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:20.971357 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c6948-1dcd-4555-bb6d-e85d58aad200 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:20.986853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8c6948-1dcd-4555-bb6d-e85d58aad200 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:20.988725 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8c6948-1dcd-4555-bb6d-e85d58aad200 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:21.007738 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-2f8c6948-1dcd-4555-bb6d-e85d58aad200 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:21.007738 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:21.007738 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:21.007738 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:21.007738 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:21.007738 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:21.007738 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:21.007738 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:21.007738 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:21.007738 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:21.007738 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:21.007738 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:21.007738 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:21.007738 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:21.007738 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:21.008898 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:21.008898 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:21.008898 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:21.008898 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:21.008898 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:21.008898 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:21.008898 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:21.008898 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:21.008898 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:21.008898 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:21.008898 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:21.008898 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:21.008898 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:21.008898 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:21.008898 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:21.008898 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:21.008898 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:21.009790 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:21.009790 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:21.009790 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:21.009790 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:21.012437 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 382/780] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:21.014126 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:21.017568 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8c6948-1dcd-4555-bb6d-e85d58aad200 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:21.032086 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c771a-838b-4811-893a-e6796d675185 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:21.072916 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad8feab-2e65-4cab-87cc-4cbdc823f30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:21.074030 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8feab-2e65-4cab-87cc-4cbdc823f30b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:21.074522 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8feab-2e65-4cab-87cc-4cbdc823f30b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:21.075067 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8feab-2e65-4cab-87cc-4cbdc823f30b None admin] PATH_INFO: `/v3/services/network` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:21.076470 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad8feab-2e65-4cab-87cc-4cbdc823f30b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:21.078650 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad8feab-2e65-4cab-87cc-4cbdc823f30b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:21.091959 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-bad8feab-2e65-4cab-87cc-4cbdc823f30b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 10:14:21.091959 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:21.091959 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 10:14:21.091959 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 10:14:21.091959 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 10:14:21.091959 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 10:14:21.091959 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 10:14:21.091959 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:21.091959 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 10:14:21.091959 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:21.091959 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 10:14:21.091959 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:21.091959 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:21.091959 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:21.091959 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:21.091959 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:21.091959 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:21.092997 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:21.092997 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:21.092997 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:21.092997 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:21.092997 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:21.092997 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:21.092997 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 10:14:21.092997 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 10:14:21.092997 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 10:14:21.092997 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 10:14:21.092997 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:21.092997 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:21.092997 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:21.092997 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:21.092997 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:21.092997 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:21.093859 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:21.093859 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:21.093859 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:21.093859 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:21.093859 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:21.093859 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:21.093859 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:21.093859 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:21.093859 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:21.093859 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:21.093859 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 10:14:21.093859 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:21.093859 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 10:14:21.093859 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:21.097233 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 399/781] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:14:20 2022] GET /identity/v3/services/network => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:14:21.099245 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a0317-9682-489c-910a-3cd5e3a17049 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:21.099528 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a0317-9682-489c-910a-3cd5e3a17049 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:21.099803 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a0317-9682-489c-910a-3cd5e3a17049 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:21.101363 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 400/782] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:21.106422 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331a0317-9682-489c-910a-3cd5e3a17049 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:21.162698 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8c6948-1dcd-4555-bb6d-e85d58aad200 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:21.212755 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3d797-850e-460b-898a-ec162b98cbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:21.214325 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3d797-850e-460b-898a-ec162b98cbac None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:21.214985 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3d797-850e-460b-898a-ec162b98cbac None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:21.215465 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3d797-850e-460b-898a-ec162b98cbac None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:21.216643 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3d797-850e-460b-898a-ec162b98cbac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:21.217545 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3d797-850e-460b-898a-ec162b98cbac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:21.219464 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3d797-850e-460b-898a-ec162b98cbac None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:21.220790 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331a0317-9682-489c-910a-3cd5e3a17049 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:21.229454 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:21.239607 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 383/783] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:14:21.241552 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aade747-5ada-4794-b65c-8500b8fa6b9f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:21.241842 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aade747-5ada-4794-b65c-8500b8fa6b9f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:21.242113 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aade747-5ada-4794-b65c-8500b8fa6b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:21.260675 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-8aade747-5ada-4794-b65c-8500b8fa6b9f None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:21.262269 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8aade747-5ada-4794-b65c-8500b8fa6b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:21.265172 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4f5c46-4d00-423a-a5e9-2bcd9157f7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:21.266950 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4f5c46-4d00-423a-a5e9-2bcd9157f7f4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:21.267493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4f5c46-4d00-423a-a5e9-2bcd9157f7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:21.267965 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4f5c46-4d00-423a-a5e9-2bcd9157f7f4 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:21.268949 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4f5c46-4d00-423a-a5e9-2bcd9157f7f4 None admin] RBAC: Adding query filter params (name=network) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:21.269492 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4f5c46-4d00-423a-a5e9-2bcd9157f7f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:21.271632 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4f5c46-4d00-423a-a5e9-2bcd9157f7f4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:21.279310 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 401/784] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 5 10:14:21 2022] GET /identity/v3/services?name=network => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:21.280742 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4f5c46-4d00-423a-a5e9-2bcd9157f7f4 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:21.304154 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:21.355621 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 384/785] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:14:21.355936 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aade747-5ada-4794-b65c-8500b8fa6b9f None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:21.428196 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4f5c46-4d00-423a-a5e9-2bcd9157f7f4 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:21.468424 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a432e0a8-a709-4da1-81bf-746be753a27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:21.469457 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a432e0a8-a709-4da1-81bf-746be753a27f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:21.470365 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a432e0a8-a709-4da1-81bf-746be753a27f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:21.470849 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a432e0a8-a709-4da1-81bf-746be753a27f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:21.481517 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a432e0a8-a709-4da1-81bf-746be753a27f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:21.482437 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aade747-5ada-4794-b65c-8500b8fa6b9f None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:21.483503 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a432e0a8-a709-4da1-81bf-746be753a27f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:21.498493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-a432e0a8-a709-4da1-81bf-746be753a27f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 10:14:21.498493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:21.498493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:21.498493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:21.498493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:21.498493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:21.498493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:21.498493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:21.498493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:21.498493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:21.498493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:21.498493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:21.498493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 10:14:21.498493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 10:14:21.498493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 10:14:21.498493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 10:14:21.499499 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:21.499499 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:21.499499 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 10:14:21.499499 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:21.499499 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 10:14:21.499499 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:21.499499 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:21.499499 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:21.499499 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:21.499499 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:21.499499 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:21.499499 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:21.499499 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:21.499499 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:21.499499 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:21.499499 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:21.499499 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:21.499499 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:21.500440 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:21.500440 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:21.500440 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 10:14:21.500440 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 10:14:21.500440 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 10:14:21.500440 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 10:14:21.500440 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 10:14:21.500440 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 10:14:21.500440 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 10:14:21.500440 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:21.502691 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 402/786] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Aug 5 10:14:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:21.504353 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a432e0a8-a709-4da1-81bf-746be753a27f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:21.517789 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:21.526630 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b05c9-2398-4ec6-9b69-7cff79d282dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:21.527412 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b05c9-2398-4ec6-9b69-7cff79d282dd None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:21.527698 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b05c9-2398-4ec6-9b69-7cff79d282dd None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:21.528048 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b05c9-2398-4ec6-9b69-7cff79d282dd None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:21.528822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858b05c9-2398-4ec6-9b69-7cff79d282dd None admin] RBAC: Adding query filter params (type=network) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:21.529085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858b05c9-2398-4ec6-9b69-7cff79d282dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:21.530569 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858b05c9-2398-4ec6-9b69-7cff79d282dd None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:21.536283 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 385/787] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 5 10:14:21 2022] GET /identity/v3/services?type=network => generated 350 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:21.537176 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b05c9-2398-4ec6-9b69-7cff79d282dd None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:21.591747 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:21.625716 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a432e0a8-a709-4da1-81bf-746be753a27f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:21.660505 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b2528c-462f-492c-8834-1cd3d8db0afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:21.661174 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2528c-462f-492c-8834-1cd3d8db0afe None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:21.661401 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2528c-462f-492c-8834-1cd3d8db0afe None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:21.661784 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2528c-462f-492c-8834-1cd3d8db0afe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:21.671735 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b05c9-2398-4ec6-9b69-7cff79d282dd None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:21.673097 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b2528c-462f-492c-8834-1cd3d8db0afe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:14:21.678505 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-63b2528c-462f-492c-8834-1cd3d8db0afe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:14:21.678505 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:21.678505 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:21.678505 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:21.678505 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:21.678505 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:21.678505 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:21.678505 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:21.678505 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:21.678505 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:21.678505 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:21.678505 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:21.678505 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:14:21.678505 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:14:21.678505 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:14:21.678505 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:14:21.679470 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:14:21.679470 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:14:21.679470 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:14:21.679470 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:14:21.679470 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:21.679470 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:21.679470 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:21.679470 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:21.679470 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:21.679470 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:21.679470 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:21.679470 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:21.679470 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:21.679470 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:21.679470 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:21.679470 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:21.680323 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:21.680323 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:21.680323 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:21.680323 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:21.680323 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:14:21.680323 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:14:21.680323 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:14:21.680323 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:14:21.680323 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:14:21.680323 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:14:21.680323 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:14:21.680323 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:21.684809 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 403/788] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 10:14:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:21.685650 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b2528c-462f-492c-8834-1cd3d8db0afe None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:21.723361 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a59b2-b12f-4b0e-aebb-1f47af9c6790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:21.724706 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a59b2-b12f-4b0e-aebb-1f47af9c6790 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:21.726058 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a59b2-b12f-4b0e-aebb-1f47af9c6790 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:21.726545 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a59b2-b12f-4b0e-aebb-1f47af9c6790 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:21.727579 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72a59b2-b12f-4b0e-aebb-1f47af9c6790 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:21.728094 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72a59b2-b12f-4b0e-aebb-1f47af9c6790 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:21.729766 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72a59b2-b12f-4b0e-aebb-1f47af9c6790 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:21.751136 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 386/789] 10.222.0.40 () {56 vars in 1149 bytes} [Fri Aug 5 10:14:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:21.752599 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a59b2-b12f-4b0e-aebb-1f47af9c6790 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:21.806784 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b2528c-462f-492c-8834-1cd3d8db0afe None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:21.811588 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:21.848060 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:21.857579 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c01833-47b3-4587-b413-f16a059e0751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:21.858992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c01833-47b3-4587-b413-f16a059e0751 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:21.859479 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c01833-47b3-4587-b413-f16a059e0751 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:21.859984 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c01833-47b3-4587-b413-f16a059e0751 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:21.860173 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a59b2-b12f-4b0e-aebb-1f47af9c6790 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:21.861204 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c01833-47b3-4587-b413-f16a059e0751 None admin] RBAC: Adding query filter params (interface=public, service_id=8107edea90b84ddaacdbea38e92c9d3c, region_id=RegionOne) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:21.862035 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c01833-47b3-4587-b413-f16a059e0751 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:21.864113 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c01833-47b3-4587-b413-f16a059e0751 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:21.872075 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 404/790] 10.222.0.40 () {56 vars in 1251 bytes} [Fri Aug 5 10:14:21 2022] GET /identity/v3/endpoints?service_id=8107edea90b84ddaacdbea38e92c9d3c&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:21.873229 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c01833-47b3-4587-b413-f16a059e0751 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:21.899501 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5641533e-bd73-48cb-ac02-bf50735a52d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:21.900042 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641533e-bd73-48cb-ac02-bf50735a52d2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:21.900317 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641533e-bd73-48cb-ac02-bf50735a52d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:21.900585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641533e-bd73-48cb-ac02-bf50735a52d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:21.901298 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5641533e-bd73-48cb-ac02-bf50735a52d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:21.901595 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5641533e-bd73-48cb-ac02-bf50735a52d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:21.903449 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5641533e-bd73-48cb-ac02-bf50735a52d2 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:21.916587 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 387/791] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:14:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:21.917610 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5641533e-bd73-48cb-ac02-bf50735a52d2 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:22.006259 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c01833-47b3-4587-b413-f16a059e0751 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:22.020883 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5641533e-bd73-48cb-ac02-bf50735a52d2 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:22.034065 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:22.054745 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d471a3-e383-4d70-904e-c6e33344b828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:22.055782 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d471a3-e383-4d70-904e-c6e33344b828 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:22.056254 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d471a3-e383-4d70-904e-c6e33344b828 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:22.056696 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d471a3-e383-4d70-904e-c6e33344b828 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:22.057485 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f029db-68f7-4247-9d49-acd50815b8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:22.058274 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f029db-68f7-4247-9d49-acd50815b8f7 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:22.058611 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f029db-68f7-4247-9d49-acd50815b8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:22.058884 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f029db-68f7-4247-9d49-acd50815b8f7 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:22.059612 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f029db-68f7-4247-9d49-acd50815b8f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:22.060981 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f029db-68f7-4247-9d49-acd50815b8f7 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:22.066608 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 388/792] 10.222.0.40 () {56 vars in 1087 bytes} [Fri Aug 5 10:14:21 2022] GET /identity/v3/services => generated 772 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:22.067510 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f029db-68f7-4247-9d49-acd50815b8f7 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:22.071609 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d471a3-e383-4d70-904e-c6e33344b828 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:22.072599 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d471a3-e383-4d70-904e-c6e33344b828 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:22.095584 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-81d471a3-e383-4d70-904e-c6e33344b828 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:22.095584 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:22.095584 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:22.095584 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:22.095584 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:22.095584 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:22.095584 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:22.095584 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:22.095584 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:22.095584 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:22.095584 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:22.095584 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:22.095584 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:22.095584 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:22.095584 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:22.096555 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:22.096555 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:22.096555 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:22.096555 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:22.096555 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:22.096555 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:22.096555 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:22.096555 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:22.096555 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:22.096555 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:22.096555 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:22.096555 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:22.096555 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:22.096555 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:22.096555 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:22.096555 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:22.096555 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:22.097464 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:22.097464 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:22.097464 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:22.097464 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:22.100960 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 405/793] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:22.108856 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d471a3-e383-4d70-904e-c6e33344b828 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:22.211016 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:22.244059 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f029db-68f7-4247-9d49-acd50815b8f7 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:22.280735 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662831fa-9f18-41fa-87bc-dac297fcfd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:22.281458 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662831fa-9f18-41fa-87bc-dac297fcfd68 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:22.281735 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662831fa-9f18-41fa-87bc-dac297fcfd68 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:22.282004 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662831fa-9f18-41fa-87bc-dac297fcfd68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:22.288117 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d471a3-e383-4d70-904e-c6e33344b828 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:22.296937 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662831fa-9f18-41fa-87bc-dac297fcfd68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:22.298471 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662831fa-9f18-41fa-87bc-dac297fcfd68 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:22.315905 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-662831fa-9f18-41fa-87bc-dac297fcfd68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:22.315905 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:22.315905 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:22.315905 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:22.315905 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:22.315905 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:22.315905 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:22.315905 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:22.315905 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:22.315905 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:22.315905 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:22.315905 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:22.315905 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:22.315905 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:22.315905 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:22.316845 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:22.316845 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:22.316845 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:22.316845 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:22.316845 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:22.316845 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:22.316845 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:22.316845 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:22.316845 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:22.316845 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:22.316845 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:22.316845 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:22.316845 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:22.316845 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:22.316845 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:22.316845 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:22.316845 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:22.317735 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:22.317735 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:22.317735 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:22.317735 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:22.322850 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 389/794] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:22.323656 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662831fa-9f18-41fa-87bc-dac297fcfd68 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:22.329946 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c1a26-b834-4e4d-892d-0113cf6c2b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:22.330800 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c1a26-b834-4e4d-892d-0113cf6c2b19 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:22.331328 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c1a26-b834-4e4d-892d-0113cf6c2b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:22.331770 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c1a26-b834-4e4d-892d-0113cf6c2b19 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:22.332624 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1c1a26-b834-4e4d-892d-0113cf6c2b19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:22.333144 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1c1a26-b834-4e4d-892d-0113cf6c2b19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:22.334720 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1c1a26-b834-4e4d-892d-0113cf6c2b19 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:22.349069 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 406/795] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:22.354746 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c1a26-b834-4e4d-892d-0113cf6c2b19 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:22.355523 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:22.488467 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662831fa-9f18-41fa-87bc-dac297fcfd68 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:22.526959 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c1a26-b834-4e4d-892d-0113cf6c2b19 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:22.552936 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aad6d9-967a-4717-8700-1228227c4d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:22.553566 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aad6d9-967a-4717-8700-1228227c4d06 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:22.553840 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aad6d9-967a-4717-8700-1228227c4d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:22.554106 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aad6d9-967a-4717-8700-1228227c4d06 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:22.554799 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aad6d9-967a-4717-8700-1228227c4d06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:22.555126 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aad6d9-967a-4717-8700-1228227c4d06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:22.556566 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aad6d9-967a-4717-8700-1228227c4d06 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:22.559074 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:22.575596 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 390/796] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:22.581062 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aad6d9-967a-4717-8700-1228227c4d06 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:22.614838 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17edda3-4684-46bd-95a8-56cbb909a99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:22.615525 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17edda3-4684-46bd-95a8-56cbb909a99b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:22.615805 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17edda3-4684-46bd-95a8-56cbb909a99b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:22.616085 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17edda3-4684-46bd-95a8-56cbb909a99b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:22.627652 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17edda3-4684-46bd-95a8-56cbb909a99b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:22.629139 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17edda3-4684-46bd-95a8-56cbb909a99b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:22.649181 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-a17edda3-4684-46bd-95a8-56cbb909a99b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:22.649181 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:22.649181 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:22.649181 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:22.649181 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:22.649181 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:22.649181 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:22.649181 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:22.649181 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:22.649181 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:22.649181 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:22.649181 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:22.649181 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 10:14:22.649181 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 10:14:22.649181 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 10:14:22.650158 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 10:14:22.650158 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:22.650158 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:22.650158 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:22.650158 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:22.650158 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:22.650158 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:22.650158 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:22.650158 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:22.650158 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:22.650158 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:22.650158 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:22.650158 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:22.650158 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:22.650158 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:22.650158 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:22.650158 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:22.651078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 10:14:22.651078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 10:14:22.651078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 10:14:22.651078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 10:14:22.651078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 10:14:22.651078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 10:14:22.651078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:22.651078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:22.651078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 407/797] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:14:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:14:22.656926 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17edda3-4684-46bd-95a8-56cbb909a99b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:22.766180 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17edda3-4684-46bd-95a8-56cbb909a99b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:22.802487 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:22.826823 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aad6d9-967a-4717-8700-1228227c4d06 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:22.837888 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4b30e9-779e-4b21-ac6f-d49d5a06ca7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:22.838529 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4b30e9-779e-4b21-ac6f-d49d5a06ca7c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:22.838810 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4b30e9-779e-4b21-ac6f-d49d5a06ca7c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:22.839113 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4b30e9-779e-4b21-ac6f-d49d5a06ca7c None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:22.839963 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4b30e9-779e-4b21-ac6f-d49d5a06ca7c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:22.840426 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4b30e9-779e-4b21-ac6f-d49d5a06ca7c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:22.843719 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4b30e9-779e-4b21-ac6f-d49d5a06ca7c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:22.869439 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 408/798] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 10:14:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:22.870720 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32cee08-ef03-4dbf-a41d-11b4e0cebc93 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:22.871037 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32cee08-ef03-4dbf-a41d-11b4e0cebc93 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:22.871314 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32cee08-ef03-4dbf-a41d-11b4e0cebc93 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:22.874934 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 409/799] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:22.874934 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32cee08-ef03-4dbf-a41d-11b4e0cebc93 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:22.889473 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef4da3-63a1-4929-9154-2db442f14f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:22.890825 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef4da3-63a1-4929-9154-2db442f14f64 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:22.891383 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef4da3-63a1-4929-9154-2db442f14f64 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:22.891949 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef4da3-63a1-4929-9154-2db442f14f64 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:22.902360 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ef4da3-63a1-4929-9154-2db442f14f64 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:22.904522 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ef4da3-63a1-4929-9154-2db442f14f64 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:22.909357 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:22.920691 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-d3ef4da3-63a1-4929-9154-2db442f14f64 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 05 10:14:22.920691 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:22.920691 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:22.920691 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:22.920691 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:22.920691 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:22.920691 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:22.920691 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:22.920691 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:22.920691 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:22.920691 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:22.920691 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:22.920691 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 10:14:22.920691 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 10:14:22.920691 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 10:14:22.920691 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 10:14:22.921876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:22.921876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:22.921876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 10:14:22.921876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:22.921876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 10:14:22.921876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:22.921876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:22.921876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:22.921876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:22.921876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:22.921876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:22.921876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:22.921876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:22.921876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:22.921876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:22.921876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:22.921876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:22.921876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:22.922842 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:22.922842 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:22.922842 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 10:14:22.922842 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 10:14:22.922842 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 10:14:22.922842 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 10:14:22.922842 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 10:14:22.922842 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 10:14:22.922842 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 05 10:14:22.922842 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:22.929617 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 391/800] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 10:14:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:22.929617 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b92f3a-b819-4e16-bb91-0ced9a2c1c9b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:22.929617 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b92f3a-b819-4e16-bb91-0ced9a2c1c9b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:22.929617 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b92f3a-b819-4e16-bb91-0ced9a2c1c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:22.950331 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b8b92f3a-b819-4e16-bb91-0ced9a2c1c9b None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:22.951667 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b8b92f3a-b819-4e16-bb91-0ced9a2c1c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:22.996988 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32cee08-ef03-4dbf-a41d-11b4e0cebc93 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:23.034996 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 392/801] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:14:23.034996 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b92f3a-b819-4e16-bb91-0ced9a2c1c9b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:23.050854 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05004c01-d641-4555-8c8a-7cc0fe427753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:23.051659 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05004c01-d641-4555-8c8a-7cc0fe427753 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:23.052377 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05004c01-d641-4555-8c8a-7cc0fe427753 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:23.053005 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05004c01-d641-4555-8c8a-7cc0fe427753 None admin] PATH_INFO: `/v3/projects/36b8b505012d4d0bac54845b6799e743/users/52fc3ba90dc045c2a5b7bb730da96acc/roles/3a2b4319820a488198dabee34420cdc7` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:23.070143 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05004c01-d641-4555-8c8a-7cc0fe427753 None admin] RBAC: Authorizing `identity:create_grant(project_id=36b8b505012d4d0bac54845b6799e743, user_id=52fc3ba90dc045c2a5b7bb730da96acc, role_id=3a2b4319820a488198dabee34420cdc7)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:23.071634 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05004c01-d641-4555-8c8a-7cc0fe427753 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:23.086053 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 410/802] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 10:14:22 2022] PUT /identity/v3/projects/36b8b505012d4d0bac54845b6799e743/users/52fc3ba90dc045c2a5b7bb730da96acc/roles/3a2b4319820a488198dabee34420cdc7 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:14:23.087317 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997af28c-16a4-4e24-89fc-fcc12eca6ce2 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:23.087604 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997af28c-16a4-4e24-89fc-fcc12eca6ce2 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:23.087882 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997af28c-16a4-4e24-89fc-fcc12eca6ce2 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:23.094006 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 411/803] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:23.094859 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997af28c-16a4-4e24-89fc-fcc12eca6ce2 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:23.126799 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:23.162211 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b92f3a-b819-4e16-bb91-0ced9a2c1c9b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:23.204210 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b188541-3869-41d0-8c54-9161418759e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:23.204924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b188541-3869-41d0-8c54-9161418759e3 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:23.205209 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b188541-3869-41d0-8c54-9161418759e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:23.205487 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b188541-3869-41d0-8c54-9161418759e3 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:23.205822 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:23.206307 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b188541-3869-41d0-8c54-9161418759e3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:23.207125 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b188541-3869-41d0-8c54-9161418759e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:23.209119 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b188541-3869-41d0-8c54-9161418759e3 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:23.234841 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 393/804] 10.222.0.40 () {56 vars in 1135 bytes} [Fri Aug 5 10:14:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:23.236233 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46fb457-d809-42ba-b595-21a749dd66af None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:23.236523 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46fb457-d809-42ba-b595-21a749dd66af None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:23.236794 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46fb457-d809-42ba-b595-21a749dd66af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:23.238535 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997af28c-16a4-4e24-89fc-fcc12eca6ce2 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:23.259568 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-f46fb457-d809-42ba-b595-21a749dd66af None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:23.260946 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f46fb457-d809-42ba-b595-21a749dd66af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:23.287727 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d96bd-3f76-4907-acaf-1caf80979200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:23.288301 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d96bd-3f76-4907-acaf-1caf80979200 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:23.288576 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d96bd-3f76-4907-acaf-1caf80979200 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:23.288844 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d96bd-3f76-4907-acaf-1caf80979200 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:23.289473 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8d96bd-3f76-4907-acaf-1caf80979200 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:23.290854 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8d96bd-3f76-4907-acaf-1caf80979200 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:23.302321 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-2f8d96bd-3f76-4907-acaf-1caf80979200 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 10:14:23.302321 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:23.302321 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 10:14:23.302321 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 10:14:23.302321 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 10:14:23.302321 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 10:14:23.302321 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 10:14:23.302321 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:23.302321 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 10:14:23.302321 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:23.302321 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 10:14:23.302321 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:23.302321 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:23.302321 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:23.302321 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:23.302321 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:23.303358 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:23.303358 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:23.303358 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:23.303358 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:23.303358 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:23.303358 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:23.303358 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:23.303358 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 10:14:23.303358 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 10:14:23.303358 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 10:14:23.303358 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 10:14:23.303358 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:23.303358 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:23.303358 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:23.303358 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:23.303358 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:23.304214 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:23.304214 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:23.304214 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:23.304214 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:23.304214 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:23.304214 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:23.304214 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:23.304214 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:23.304214 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:23.304214 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:23.304214 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:23.304214 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 10:14:23.304214 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:23.304214 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 10:14:23.304214 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:23.304214 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 412/805] 10.222.0.40 () {56 vars in 1129 bytes} [Fri Aug 5 10:14:23 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:23.305059 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d96bd-3f76-4907-acaf-1caf80979200 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:23.344480 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 394/806] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:14:23.346258 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46fb457-d809-42ba-b595-21a749dd66af None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:23.429516 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:23.439701 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d96bd-3f76-4907-acaf-1caf80979200 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:23.467777 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46fb457-d809-42ba-b595-21a749dd66af None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:23.473806 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e2f74f-dd6b-4fc1-968a-14f33ae996b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:23.474462 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e2f74f-dd6b-4fc1-968a-14f33ae996b2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:23.474741 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e2f74f-dd6b-4fc1-968a-14f33ae996b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:23.475050 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e2f74f-dd6b-4fc1-968a-14f33ae996b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:23.486330 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e2f74f-dd6b-4fc1-968a-14f33ae996b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:23.487833 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e2f74f-dd6b-4fc1-968a-14f33ae996b2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:23.489641 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:23.503259 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-e7e2f74f-dd6b-4fc1-968a-14f33ae996b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:23.503259 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:23.503259 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:23.503259 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:23.503259 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:23.503259 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:23.503259 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:23.503259 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:23.503259 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:23.503259 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:23.503259 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:23.503259 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:23.503259 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:23.503259 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:23.503259 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:23.504344 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:23.504344 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:23.504344 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:23.504344 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:23.504344 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:23.504344 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:23.504344 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:23.504344 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:23.504344 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:23.504344 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:23.504344 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:23.504344 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:23.504344 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:23.504344 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:23.504344 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:23.504344 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:23.504344 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:23.505259 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:23.505259 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:23.505259 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:23.505259 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:23.506971 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 413/807] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:23.507939 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e2f74f-dd6b-4fc1-968a-14f33ae996b2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:23.514810 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe73aa3d-5716-4f6e-9c0c-62ed08570a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:23.515668 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe73aa3d-5716-4f6e-9c0c-62ed08570a26 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:23.516086 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe73aa3d-5716-4f6e-9c0c-62ed08570a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:23.516620 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe73aa3d-5716-4f6e-9c0c-62ed08570a26 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:23.518589 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe73aa3d-5716-4f6e-9c0c-62ed08570a26 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:23.519060 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe73aa3d-5716-4f6e-9c0c-62ed08570a26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:23.520586 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe73aa3d-5716-4f6e-9c0c-62ed08570a26 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:23.530658 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 395/808] 10.222.0.40 () {56 vars in 1124 bytes} [Fri Aug 5 10:14:23 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:23.531791 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe73aa3d-5716-4f6e-9c0c-62ed08570a26 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:23.616753 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e2f74f-dd6b-4fc1-968a-14f33ae996b2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:23.635514 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe73aa3d-5716-4f6e-9c0c-62ed08570a26 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:23.652257 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf71d71-7a45-448d-a3ca-bd987285a771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:23.653204 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf71d71-7a45-448d-a3ca-bd987285a771 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:23.653626 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf71d71-7a45-448d-a3ca-bd987285a771 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:23.654064 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf71d71-7a45-448d-a3ca-bd987285a771 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:23.665516 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf71d71-7a45-448d-a3ca-bd987285a771 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:14:23.666090 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:23.673263 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20829da0-fff2-4297-9d17-8ecd3ee85daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:23.674067 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20829da0-fff2-4297-9d17-8ecd3ee85daf None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:23.674486 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20829da0-fff2-4297-9d17-8ecd3ee85daf None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:23.674597 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-bbf71d71-7a45-448d-a3ca-bd987285a771 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:14:23.674597 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:23.674597 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:23.674597 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:23.674597 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:23.674597 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:23.674597 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:23.674597 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:23.674597 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:23.674597 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:23.674597 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:23.674597 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:23.674597 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:14:23.674597 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:14:23.674597 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:14:23.674597 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:14:23.675512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:14:23.675512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:14:23.675512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:14:23.675512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:14:23.675512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:23.675512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:23.675512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:23.675512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:23.675512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:23.675512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:23.675512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:23.675512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:23.675512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:23.675512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:23.675512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:23.675512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:23.676430 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:23.676430 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:23.676430 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:23.676430 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:23.676430 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:14:23.676430 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:14:23.676430 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:14:23.676430 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:14:23.676430 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:14:23.676430 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:14:23.676430 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:14:23.676430 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:23.676430 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 414/809] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 10:14:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:23.677103 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf71d71-7a45-448d-a3ca-bd987285a771 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:23.677995 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20829da0-fff2-4297-9d17-8ecd3ee85daf None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:23.678864 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20829da0-fff2-4297-9d17-8ecd3ee85daf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:23.679386 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20829da0-fff2-4297-9d17-8ecd3ee85daf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:23.681123 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20829da0-fff2-4297-9d17-8ecd3ee85daf None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:23.693789 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 396/810] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:23.694878 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20829da0-fff2-4297-9d17-8ecd3ee85daf None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:23.742340 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:23.791613 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf71d71-7a45-448d-a3ca-bd987285a771 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:23.799909 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20829da0-fff2-4297-9d17-8ecd3ee85daf None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:23.840142 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbce77b-d2bb-4c62-b8fa-c03383235eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:23.841106 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbce77b-d2bb-4c62-b8fa-c03383235eff None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:23.841523 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbce77b-d2bb-4c62-b8fa-c03383235eff None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:23.841948 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699d8b14-a5a4-443b-a5bc-72832fe9271a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:23.842443 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbce77b-d2bb-4c62-b8fa-c03383235eff None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:23.842714 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d8b14-a5a4-443b-a5bc-72832fe9271a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:23.843087 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d8b14-a5a4-443b-a5bc-72832fe9271a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:23.843428 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d8b14-a5a4-443b-a5bc-72832fe9271a None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:23.843835 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbce77b-d2bb-4c62-b8fa-c03383235eff None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:23.844273 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbce77b-d2bb-4c62-b8fa-c03383235eff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:23.844388 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699d8b14-a5a4-443b-a5bc-72832fe9271a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:23.844703 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699d8b14-a5a4-443b-a5bc-72832fe9271a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:23.846036 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbce77b-d2bb-4c62-b8fa-c03383235eff None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:23.846186 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699d8b14-a5a4-443b-a5bc-72832fe9271a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:23.857937 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 415/811] 10.222.0.40 () {56 vars in 1124 bytes} [Fri Aug 5 10:14:23 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:23.859385 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbce77b-d2bb-4c62-b8fa-c03383235eff None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:23.862204 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 397/812] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:14:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:23.863483 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699d8b14-a5a4-443b-a5bc-72832fe9271a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:23.932170 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:23.981935 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbce77b-d2bb-4c62-b8fa-c03383235eff None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:23.994083 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699d8b14-a5a4-443b-a5bc-72832fe9271a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:24.017447 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:24.025258 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2776169-f330-4aac-abef-ff2f9c812ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:24.026479 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2776169-f330-4aac-abef-ff2f9c812ab7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:24.027239 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2776169-f330-4aac-abef-ff2f9c812ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:24.027677 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2776169-f330-4aac-abef-ff2f9c812ab7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:24.034304 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c16eb3-6c9a-4095-b426-b15a1ad07126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:24.035237 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c16eb3-6c9a-4095-b426-b15a1ad07126 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:24.035693 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c16eb3-6c9a-4095-b426-b15a1ad07126 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:24.036058 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c16eb3-6c9a-4095-b426-b15a1ad07126 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:24.036353 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2776169-f330-4aac-abef-ff2f9c812ab7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:24.036961 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c16eb3-6c9a-4095-b426-b15a1ad07126 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:24.038089 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2776169-f330-4aac-abef-ff2f9c812ab7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:24.038509 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c16eb3-6c9a-4095-b426-b15a1ad07126 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:24.052360 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-a2776169-f330-4aac-abef-ff2f9c812ab7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:24.052360 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:24.052360 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:24.052360 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:24.052360 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:24.052360 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:24.052360 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:24.052360 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:24.052360 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:24.052360 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:24.052360 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:24.052360 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:24.052360 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 10:14:24.052360 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 10:14:24.052360 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 10:14:24.053683 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 10:14:24.053683 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:24.053683 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:24.053683 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:24.053683 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:24.053683 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:24.053683 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:24.053683 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:24.053683 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:24.053683 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:24.053683 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:24.053683 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:24.053683 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:24.053683 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:24.053683 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:24.053683 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:24.053683 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:24.054614 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 10:14:24.054614 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 10:14:24.054614 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 10:14:24.054614 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 10:14:24.054614 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 10:14:24.054614 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 10:14:24.054614 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:24.054614 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:24.056774 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 416/813] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:14:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:24.057767 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2776169-f330-4aac-abef-ff2f9c812ab7 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:24.074927 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 398/814] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:14:23 2022] POST /identity/v3/endpoints => generated 346 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:14:24.074927 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c16eb3-6c9a-4095-b426-b15a1ad07126 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:24.272212 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2776169-f330-4aac-abef-ff2f9c812ab7 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:24.285104 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c16eb3-6c9a-4095-b426-b15a1ad07126 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:24.288386 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:24.317458 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ddda3-1dd9-48ea-b268-c9329b511578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:24.318410 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ddda3-1dd9-48ea-b268-c9329b511578 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:24.324256 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ddda3-1dd9-48ea-b268-c9329b511578 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:24.325357 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ddda3-1dd9-48ea-b268-c9329b511578 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:24.335090 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2bd38-54ab-429b-9f28-1479b2403910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:24.336203 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2bd38-54ab-429b-9f28-1479b2403910 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:24.336203 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2bd38-54ab-429b-9f28-1479b2403910 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:24.336373 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2bd38-54ab-429b-9f28-1479b2403910 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:24.337164 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c2bd38-54ab-429b-9f28-1479b2403910 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:24.337273 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c2bd38-54ab-429b-9f28-1479b2403910 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:24.338703 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c2bd38-54ab-429b-9f28-1479b2403910 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:24.342694 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35ddda3-1dd9-48ea-b268-c9329b511578 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:24.344207 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35ddda3-1dd9-48ea-b268-c9329b511578 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:24.355682 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 399/815] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 10:14:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:24.357232 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1264774-d260-493b-90cf-966cb242ff40 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:24.357595 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1264774-d260-493b-90cf-966cb242ff40 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:24.357943 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1264774-d260-493b-90cf-966cb242ff40 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:24.359954 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 400/816] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:24.361646 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-e35ddda3-1dd9-48ea-b268-c9329b511578 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:24.361646 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:24.361646 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:24.361646 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:24.361646 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:24.361646 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:24.361646 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:24.361646 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:24.361646 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:24.361646 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:24.361646 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:24.361646 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:24.361646 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:24.361646 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:24.361646 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:24.362510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:24.362510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:24.362510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:24.362510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:24.362510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:24.362510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:24.362510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:24.362510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:24.362510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:24.362510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:24.362510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:24.362510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:24.362510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:24.362510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:24.362510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:24.362510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:24.362510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:24.363784 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:24.363784 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:24.363784 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:24.363784 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:24.363784 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 417/817] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:24.364656 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1264774-d260-493b-90cf-966cb242ff40 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:24.366202 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031177d9-95b8-4f88-936f-c622f039b1e3 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:24.366489 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031177d9-95b8-4f88-936f-c622f039b1e3 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:24.366768 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031177d9-95b8-4f88-936f-c622f039b1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:24.387978 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-031177d9-95b8-4f88-936f-c622f039b1e3 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:24.389156 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-031177d9-95b8-4f88-936f-c622f039b1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:24.415228 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:24.474371 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 418/818] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:14:24.475592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031177d9-95b8-4f88-936f-c622f039b1e3 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:24.510634 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1264774-d260-493b-90cf-966cb242ff40 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:24.556112 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c715fea7-4268-4336-86d9-5550042012f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:24.556713 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715fea7-4268-4336-86d9-5550042012f3 None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:24.556996 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715fea7-4268-4336-86d9-5550042012f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:24.557271 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715fea7-4268-4336-86d9-5550042012f3 None admin] PATH_INFO: `/v3/projects/36b8b505012d4d0bac54845b6799e743/users/c80cf0e429f04f229aea1aa9293a93b2/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:24.559596 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c715fea7-4268-4336-86d9-5550042012f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=36b8b505012d4d0bac54845b6799e743, user_id=c80cf0e429f04f229aea1aa9293a93b2, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:24.561990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c715fea7-4268-4336-86d9-5550042012f3 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:24.586179 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 401/819] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 10:14:24 2022] PUT /identity/v3/projects/36b8b505012d4d0bac54845b6799e743/users/c80cf0e429f04f229aea1aa9293a93b2/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:14:24.587148 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c715fea7-4268-4336-86d9-5550042012f3 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:24.607434 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031177d9-95b8-4f88-936f-c622f039b1e3 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:24.622681 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:24.681970 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2c8e88-3a11-408f-aedf-2613548100f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:24.682946 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c8e88-3a11-408f-aedf-2613548100f9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:24.683485 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c8e88-3a11-408f-aedf-2613548100f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:24.683956 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c8e88-3a11-408f-aedf-2613548100f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:24.684805 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2c8e88-3a11-408f-aedf-2613548100f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:24.685276 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2c8e88-3a11-408f-aedf-2613548100f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:24.686889 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2c8e88-3a11-408f-aedf-2613548100f9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:24.713548 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 419/820] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:24.714959 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c715fea7-4268-4336-86d9-5550042012f3 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:24.719722 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2c8e88-3a11-408f-aedf-2613548100f9 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:24.756439 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c3e38-827c-4748-a0dd-5f4cc8090a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:24.757111 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c3e38-827c-4748-a0dd-5f4cc8090a1b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:24.757395 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c3e38-827c-4748-a0dd-5f4cc8090a1b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:24.757669 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c3e38-827c-4748-a0dd-5f4cc8090a1b None admin] PATH_INFO: `/v3/services/network` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:24.758311 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33c3e38-827c-4748-a0dd-5f4cc8090a1b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:24.759746 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33c3e38-827c-4748-a0dd-5f4cc8090a1b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:24.772332 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:24.780759 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-a33c3e38-827c-4748-a0dd-5f4cc8090a1b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 10:14:24.780759 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:24.780759 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 10:14:24.780759 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 10:14:24.780759 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 10:14:24.780759 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 10:14:24.780759 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 10:14:24.780759 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:24.780759 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 10:14:24.780759 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:24.780759 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 10:14:24.780759 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:24.780759 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:24.780759 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:24.780759 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:24.780759 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:24.780759 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:24.782093 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:24.782093 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:24.782093 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:24.782093 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:24.782093 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:24.782093 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:24.782093 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 10:14:24.782093 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 10:14:24.782093 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 10:14:24.782093 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 10:14:24.782093 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:24.782093 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:24.782093 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:24.782093 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:24.782093 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:24.782093 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:24.783183 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:24.783183 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:24.783183 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:24.783183 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:24.783183 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:24.783183 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:24.783183 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:24.783183 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:24.783183 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:24.783183 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:24.783183 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 10:14:24.783183 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:24.783183 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 10:14:24.783183 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:24.790502 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 402/821] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:14:24 2022] GET /identity/v3/services/network => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:24.801797 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c3e38-827c-4748-a0dd-5f4cc8090a1b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:24.904930 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2c8e88-3a11-408f-aedf-2613548100f9 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:24.922113 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:24.933339 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c3e38-827c-4748-a0dd-5f4cc8090a1b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:24.947428 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d9265-602c-4b78-92bd-30f98696805b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:24.948357 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d9265-602c-4b78-92bd-30f98696805b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:24.948873 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d9265-602c-4b78-92bd-30f98696805b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:24.949525 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d9265-602c-4b78-92bd-30f98696805b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:24.959359 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07d9265-602c-4b78-92bd-30f98696805b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:24.961011 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07d9265-602c-4b78-92bd-30f98696805b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:24.973445 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5699f0f3-4a3b-49a4-92f5-de213c5c82dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:24.974073 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699f0f3-4a3b-49a4-92f5-de213c5c82dd None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:24.974353 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699f0f3-4a3b-49a4-92f5-de213c5c82dd None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:24.974623 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699f0f3-4a3b-49a4-92f5-de213c5c82dd None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:24.974795 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-d07d9265-602c-4b78-92bd-30f98696805b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 10:14:24.974795 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:24.974795 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:24.974795 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:24.974795 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:24.974795 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:24.974795 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:24.974795 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:24.974795 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:24.974795 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:24.974795 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:24.974795 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:24.974795 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 10:14:24.974795 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 10:14:24.974795 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 10:14:24.974795 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 10:14:24.975928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:24.975928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:24.975928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 10:14:24.975928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:24.975928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 10:14:24.975928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:24.975928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:24.975928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:24.975928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:24.975928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:24.975928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:24.975928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:24.975928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:24.975928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:24.975928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:24.975928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:24.975928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:24.975928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:24.977076 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:24.977076 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:24.977076 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 10:14:24.977076 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 10:14:24.977076 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 10:14:24.977076 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 10:14:24.977076 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 10:14:24.977076 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 10:14:24.977076 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 10:14:24.977076 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:24.977621 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5699f0f3-4a3b-49a4-92f5-de213c5c82dd None admin] RBAC: Adding query filter params (name=network) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:24.977621 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5699f0f3-4a3b-49a4-92f5-de213c5c82dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:24.977621 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5699f0f3-4a3b-49a4-92f5-de213c5c82dd None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:24.980056 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 420/822] 10.222.0.40 () {56 vars in 1093 bytes} [Fri Aug 5 10:14:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:14:24.982971 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 403/823] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 5 10:14:24 2022] GET /identity/v3/services?name=network => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:14:24.987287 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d9265-602c-4b78-92bd-30f98696805b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:24.995093 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5699f0f3-4a3b-49a4-92f5-de213c5c82dd None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:25.118156 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:25.175091 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d9265-602c-4b78-92bd-30f98696805b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:25.190990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5699f0f3-4a3b-49a4-92f5-de213c5c82dd None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:25.236166 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd558605-ab98-4a3f-86d2-6659ba756fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:25.236805 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd558605-ab98-4a3f-86d2-6659ba756fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:25.237084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd558605-ab98-4a3f-86d2-6659ba756fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:25.237358 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd558605-ab98-4a3f-86d2-6659ba756fb0 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:25.238161 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd558605-ab98-4a3f-86d2-6659ba756fb0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:25.238472 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd558605-ab98-4a3f-86d2-6659ba756fb0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:25.240058 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd558605-ab98-4a3f-86d2-6659ba756fb0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:25.261350 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a53d08-bea3-435b-8eb2-a6bc0f97341b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:25.261594 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 421/824] 10.222.0.40 () {56 vars in 1133 bytes} [Fri Aug 5 10:14:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:25.262556 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a53d08-bea3-435b-8eb2-a6bc0f97341b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:25.262839 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a53d08-bea3-435b-8eb2-a6bc0f97341b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:25.263136 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a53d08-bea3-435b-8eb2-a6bc0f97341b None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:25.263830 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a53d08-bea3-435b-8eb2-a6bc0f97341b None admin] RBAC: Adding query filter params (type=network) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:25.264133 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a53d08-bea3-435b-8eb2-a6bc0f97341b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:25.265547 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a53d08-bea3-435b-8eb2-a6bc0f97341b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:25.267179 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd558605-ab98-4a3f-86d2-6659ba756fb0 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:25.277556 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 404/825] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 5 10:14:24 2022] GET /identity/v3/services?type=network => generated 350 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:25.284811 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a53d08-bea3-435b-8eb2-a6bc0f97341b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:25.320699 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:25.403479 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd558605-ab98-4a3f-86d2-6659ba756fb0 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:25.413453 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a53d08-bea3-435b-8eb2-a6bc0f97341b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:25.441103 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c564ee83-9698-43d8-8140-69e58b7ffdf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:25.441955 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564ee83-9698-43d8-8140-69e58b7ffdf0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:25.442379 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564ee83-9698-43d8-8140-69e58b7ffdf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:25.442931 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564ee83-9698-43d8-8140-69e58b7ffdf0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:25.451662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960569ed-dfd8-4f66-9af0-bdfd22e90e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:25.452570 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:25.453326 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960569ed-dfd8-4f66-9af0-bdfd22e90e30 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:25.453991 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960569ed-dfd8-4f66-9af0-bdfd22e90e30 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:25.454666 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960569ed-dfd8-4f66-9af0-bdfd22e90e30 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:25.456051 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960569ed-dfd8-4f66-9af0-bdfd22e90e30 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:25.457670 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960569ed-dfd8-4f66-9af0-bdfd22e90e30 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:25.484361 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 405/826] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:14:25 2022] POST /identity/v3/endpoints => generated 337 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:14:25.485456 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2951f-8c90-441e-b89c-9a041b2ccb3c None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:25.485746 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2951f-8c90-441e-b89c-9a041b2ccb3c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:25.486024 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2951f-8c90-441e-b89c-9a041b2ccb3c None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:25.499683 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 406/827] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:25 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:25.508698 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e00c76-e447-4401-9ddd-b2b38a5fd55b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:25.509034 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e00c76-e447-4401-9ddd-b2b38a5fd55b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:25.509256 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e00c76-e447-4401-9ddd-b2b38a5fd55b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:25.532720 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c564ee83-9698-43d8-8140-69e58b7ffdf0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:25.535408 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c564ee83-9698-43d8-8140-69e58b7ffdf0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:25.545634 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-d5e00c76-e447-4401-9ddd-b2b38a5fd55b None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:25.546837 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d5e00c76-e447-4401-9ddd-b2b38a5fd55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:25.648165 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 422/828] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:14:25 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 10:14:25.658010 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c564ee83-9698-43d8-8140-69e58b7ffdf0 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:25.670964 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:25.694569 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 407/829] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:25 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 10:14:25.716423 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:25.722361 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e00c76-e447-4401-9ddd-b2b38a5fd55b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:25.806861 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c564ee83-9698-43d8-8140-69e58b7ffdf0 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:25.849222 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e00c76-e447-4401-9ddd-b2b38a5fd55b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:25.850411 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6be12f-c308-414c-9f32-872b2d40c5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:25.851355 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6be12f-c308-414c-9f32-872b2d40c5a7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:25.851638 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6be12f-c308-414c-9f32-872b2d40c5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:25.851912 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6be12f-c308-414c-9f32-872b2d40c5a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:25.866533 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6be12f-c308-414c-9f32-872b2d40c5a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:25.868823 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6be12f-c308-414c-9f32-872b2d40c5a7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:25.890607 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-fb6be12f-c308-414c-9f32-872b2d40c5a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:25.890607 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:25.890607 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:25.890607 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:25.890607 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:25.890607 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:25.890607 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:25.890607 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:25.890607 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:25.890607 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:25.890607 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:25.890607 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:25.890607 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:25.890607 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:25.890607 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:25.891592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:25.891592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:25.891592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:25.891592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:25.891592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:25.891592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:25.891592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:25.891592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:25.891592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:25.891592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:25.891592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:25.891592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:25.891592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:25.891592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:25.891592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:25.891592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:25.891592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:25.893073 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:25.893073 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:25.893073 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:25.893073 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:25.893073 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 423/830] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:25.893417 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a544ddd7-0a0e-4781-8f83-0df6ad6c31f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:25.893588 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544ddd7-0a0e-4781-8f83-0df6ad6c31f0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:25.894391 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544ddd7-0a0e-4781-8f83-0df6ad6c31f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:25.894391 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544ddd7-0a0e-4781-8f83-0df6ad6c31f0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:25.897322 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6be12f-c308-414c-9f32-872b2d40c5a7 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:25.909230 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a544ddd7-0a0e-4781-8f83-0df6ad6c31f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:14:25.916260 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-a544ddd7-0a0e-4781-8f83-0df6ad6c31f0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 10:14:25.916260 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:25.916260 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:25.916260 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:25.916260 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:25.916260 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:25.916260 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:25.916260 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:25.916260 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:25.916260 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:25.916260 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:25.916260 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:25.916260 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:14:25.916260 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:14:25.916260 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:14:25.916260 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:14:25.917451 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:14:25.917451 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:14:25.917451 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:14:25.917451 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:14:25.917451 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:25.917451 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:25.917451 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:25.917451 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:25.917451 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:25.917451 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:25.917451 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:25.917451 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:25.917451 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:25.917451 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:25.917451 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:25.917451 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:25.919284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:25.919284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:25.919284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:25.919284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:25.919284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:14:25.919284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:14:25.919284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:14:25.919284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:14:25.919284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:14:25.919284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:14:25.919284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 10:14:25.919284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:25.919284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 408/831] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 10:14:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:25.924551 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a544ddd7-0a0e-4781-8f83-0df6ad6c31f0 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:25.994086 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:26.020712 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:26.030604 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6be12f-c308-414c-9f32-872b2d40c5a7 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:26.042094 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a544ddd7-0a0e-4781-8f83-0df6ad6c31f0 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:26.077229 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc200137-7bca-45e5-8d87-c4c1f5ea7b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:26.078109 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc200137-7bca-45e5-8d87-c4c1f5ea7b91 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:26.078555 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc200137-7bca-45e5-8d87-c4c1f5ea7b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:26.078927 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc200137-7bca-45e5-8d87-c4c1f5ea7b91 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:26.079541 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c8acc-315a-4806-b1a0-88580d87be49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:26.079845 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc200137-7bca-45e5-8d87-c4c1f5ea7b91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:26.080170 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c8acc-315a-4806-b1a0-88580d87be49 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:26.080344 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc200137-7bca-45e5-8d87-c4c1f5ea7b91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:26.080448 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c8acc-315a-4806-b1a0-88580d87be49 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:26.080711 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c8acc-315a-4806-b1a0-88580d87be49 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:26.081431 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c8acc-315a-4806-b1a0-88580d87be49 None admin] RBAC: Adding query filter params (name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:26.081735 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c8acc-315a-4806-b1a0-88580d87be49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:26.082148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc200137-7bca-45e5-8d87-c4c1f5ea7b91 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:26.083135 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c8acc-315a-4806-b1a0-88580d87be49 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:26.096990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 409/833] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 5 10:14:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:14:26.098984 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 424/833] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:14:26.102090 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c8acc-315a-4806-b1a0-88580d87be49 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:26.103149 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc200137-7bca-45e5-8d87-c4c1f5ea7b91 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:26.225805 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc200137-7bca-45e5-8d87-c4c1f5ea7b91 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:26.230055 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c8acc-315a-4806-b1a0-88580d87be49 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:26.251537 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:26.268141 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eee95a-c972-4e09-b49e-21beab298437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:26.269087 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eee95a-c972-4e09-b49e-21beab298437 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:26.269477 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eee95a-c972-4e09-b49e-21beab298437 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:26.269832 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eee95a-c972-4e09-b49e-21beab298437 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:26.272472 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca72da99-94f5-4907-834c-4e9bbee5796f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:26.273666 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72da99-94f5-4907-834c-4e9bbee5796f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:26.274084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72da99-94f5-4907-834c-4e9bbee5796f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:26.274493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72da99-94f5-4907-834c-4e9bbee5796f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:26.281738 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eee95a-c972-4e09-b49e-21beab298437 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:26.283335 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca72da99-94f5-4907-834c-4e9bbee5796f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:26.283454 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eee95a-c972-4e09-b49e-21beab298437 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:26.285008 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca72da99-94f5-4907-834c-4e9bbee5796f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:26.299029 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-ca72da99-94f5-4907-834c-4e9bbee5796f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:26.299029 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:26.299029 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:26.299029 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:26.299029 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:26.299029 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:26.299029 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:26.299029 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:26.299029 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:26.299029 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:26.299029 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:26.299029 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:26.299029 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 10:14:26.299029 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 10:14:26.299029 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 10:14:26.300132 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 10:14:26.300132 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:26.300132 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:26.300132 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:26.300132 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:26.300132 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:26.300132 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:26.300132 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:26.300132 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:26.300132 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:26.300132 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:26.300132 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:26.300132 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:26.300132 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:26.300132 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:26.300132 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:26.300132 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:26.301227 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 10:14:26.301227 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 10:14:26.301227 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 10:14:26.301227 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 10:14:26.301227 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 10:14:26.301227 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 10:14:26.301227 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:26.301227 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:26.301769 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-d3eee95a-c972-4e09-b49e-21beab298437 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:26.301769 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:26.301769 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:26.301769 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:26.301769 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:26.301769 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:26.301769 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:26.301769 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:26.301769 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:26.301769 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:26.301769 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:26.301769 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:26.301769 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:26.301769 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:26.301769 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:26.302746 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:26.302746 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:26.302746 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:26.302746 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:26.302746 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:26.302746 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:26.302746 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:26.302746 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:26.302746 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:26.302746 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:26.302746 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:26.302746 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:26.302746 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:26.302746 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:26.302746 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:26.302746 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:26.302746 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:26.303860 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:26.303860 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:26.303860 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:26.303860 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:26.310136 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 410/835] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:26.310311 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 425/835] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:14:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:26.311249 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eee95a-c972-4e09-b49e-21beab298437 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:26.311456 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca72da99-94f5-4907-834c-4e9bbee5796f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:26.327578 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:26.429184 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eee95a-c972-4e09-b49e-21beab298437 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:26.429398 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca72da99-94f5-4907-834c-4e9bbee5796f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:26.467683 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d7b451-bdf5-4652-8a11-6003cc385338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:26.468308 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7b451-bdf5-4652-8a11-6003cc385338 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:26.468587 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7b451-bdf5-4652-8a11-6003cc385338 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:26.468696 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd02a87c-a21c-4e9d-94fe-b7c2c8d15d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:26.468849 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7b451-bdf5-4652-8a11-6003cc385338 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:26.469581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02a87c-a21c-4e9d-94fe-b7c2c8d15d58 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:26.469694 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d7b451-bdf5-4652-8a11-6003cc385338 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:26.469882 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d7b451-bdf5-4652-8a11-6003cc385338 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:26.470238 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02a87c-a21c-4e9d-94fe-b7c2c8d15d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:26.470642 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02a87c-a21c-4e9d-94fe-b7c2c8d15d58 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:26.471356 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d7b451-bdf5-4652-8a11-6003cc385338 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:26.471670 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd02a87c-a21c-4e9d-94fe-b7c2c8d15d58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:26.472122 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd02a87c-a21c-4e9d-94fe-b7c2c8d15d58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:26.473622 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd02a87c-a21c-4e9d-94fe-b7c2c8d15d58 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:26.483994 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 411/836] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 10:14:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:26.485378 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b125e5a4-d941-4c84-a106-bae5ee08935b None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:26.485660 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b125e5a4-d941-4c84-a106-bae5ee08935b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:26.485929 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b125e5a4-d941-4c84-a106-bae5ee08935b None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:26.487318 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 412/837] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:26.491276 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 426/838] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:26.492068 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b125e5a4-d941-4c84-a106-bae5ee08935b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:26.492783 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2706339f-9680-413a-bbe1-78bf0cab2c8f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:26.493066 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2706339f-9680-413a-bbe1-78bf0cab2c8f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:26.493407 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2706339f-9680-413a-bbe1-78bf0cab2c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:26.511809 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-2706339f-9680-413a-bbe1-78bf0cab2c8f None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:26.512904 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2706339f-9680-413a-bbe1-78bf0cab2c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:26.532730 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:26.581348 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:26.590051 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 427/839] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:26 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 10:14:26.590951 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2706339f-9680-413a-bbe1-78bf0cab2c8f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:26.644742 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b125e5a4-d941-4c84-a106-bae5ee08935b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:26.682532 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319e40e-7a79-4887-be8f-cfdd12716294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:26.683289 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319e40e-7a79-4887-be8f-cfdd12716294 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:26.683574 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319e40e-7a79-4887-be8f-cfdd12716294 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:26.683821 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319e40e-7a79-4887-be8f-cfdd12716294 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:26.684715 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3319e40e-7a79-4887-be8f-cfdd12716294 None admin] RBAC: Adding query filter params (role.id=daef89adcaeb48a382a0bcb03340ccbb, scope.project.id=36b8b505012d4d0bac54845b6799e743, user.id=b938f1f41db94b0eafe95bc8eeaae175) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:26.685023 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3319e40e-7a79-4887-be8f-cfdd12716294 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:26.686456 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3319e40e-7a79-4887-be8f-cfdd12716294 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:26.703124 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 413/840] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:14:26 2022] GET /identity/v3/role_assignments?user.id=b938f1f41db94b0eafe95bc8eeaae175&scope.project.id=36b8b505012d4d0bac54845b6799e743&role.id=daef89adcaeb48a382a0bcb03340ccbb => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:26.703124 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319e40e-7a79-4887-be8f-cfdd12716294 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:26.708437 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2706339f-9680-413a-bbe1-78bf0cab2c8f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:26.758314 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d684176-dc15-46a9-a776-364f65cce356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:26.759199 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d684176-dc15-46a9-a776-364f65cce356 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:26.759546 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d684176-dc15-46a9-a776-364f65cce356 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:26.759933 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d684176-dc15-46a9-a776-364f65cce356 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:26.771629 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d684176-dc15-46a9-a776-364f65cce356 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:26.773424 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d684176-dc15-46a9-a776-364f65cce356 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:26.790721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-0d684176-dc15-46a9-a776-364f65cce356 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 10:14:26.790721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:26.790721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:26.790721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:26.790721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:26.790721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:26.790721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:26.790721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:26.790721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:26.790721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:26.790721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:26.790721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:26.790721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 10:14:26.790721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 10:14:26.790721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 10:14:26.790721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 10:14:26.791793 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:26.791793 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:26.791793 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 10:14:26.791793 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:26.791793 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 10:14:26.791793 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:26.791793 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:26.791793 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:26.791793 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:26.791793 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:26.791793 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:26.791793 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:26.791793 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:26.791793 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:26.791793 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:26.791793 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:26.791793 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:26.791793 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:26.793155 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:26.793155 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:26.793155 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 10:14:26.793155 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 10:14:26.793155 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 10:14:26.793155 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 10:14:26.793155 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 10:14:26.793155 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 10:14:26.793155 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 10:14:26.793155 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:26.795080 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 428/841] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Aug 5 10:14:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:26.799575 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d684176-dc15-46a9-a776-364f65cce356 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:26.814429 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:26.824977 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319e40e-7a79-4887-be8f-cfdd12716294 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:26.865953 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10239b64-774b-4318-a3a4-2974c6aa49f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:26.867001 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10239b64-774b-4318-a3a4-2974c6aa49f2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:26.867514 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10239b64-774b-4318-a3a4-2974c6aa49f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:26.867880 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10239b64-774b-4318-a3a4-2974c6aa49f2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:26.868724 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10239b64-774b-4318-a3a4-2974c6aa49f2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:26.870253 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10239b64-774b-4318-a3a4-2974c6aa49f2 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:26.880393 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-10239b64-774b-4318-a3a4-2974c6aa49f2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 10:14:26.880393 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:26.880393 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 10:14:26.880393 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 10:14:26.880393 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 10:14:26.880393 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 10:14:26.880393 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 10:14:26.880393 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:26.880393 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 10:14:26.880393 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:26.880393 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 10:14:26.880393 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:26.880393 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:26.880393 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:26.880393 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:26.880393 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:26.880393 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:26.881653 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:26.881653 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:26.881653 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:26.881653 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:26.881653 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:26.881653 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:26.881653 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 10:14:26.881653 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 10:14:26.881653 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 10:14:26.881653 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 10:14:26.881653 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:26.881653 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:26.881653 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:26.881653 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:26.881653 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:26.881653 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:26.882709 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:26.882709 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:26.882709 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:26.882709 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:26.882709 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:26.882709 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:26.882709 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:26.882709 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:26.882709 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:26.882709 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:26.882709 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 10:14:26.882709 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:26.882709 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 10:14:26.882709 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:26.884700 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 414/842] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 5 10:14:26 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:26.890497 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10239b64-774b-4318-a3a4-2974c6aa49f2 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:26.892729 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:26.942652 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d684176-dc15-46a9-a776-364f65cce356 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:26.979631 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7bf1e4-f957-4f2e-bd3f-771989239d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:26.980317 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bf1e4-f957-4f2e-bd3f-771989239d72 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:26.980596 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bf1e4-f957-4f2e-bd3f-771989239d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:26.980867 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bf1e4-f957-4f2e-bd3f-771989239d72 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:26.981673 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7bf1e4-f957-4f2e-bd3f-771989239d72 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:26.981981 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7bf1e4-f957-4f2e-bd3f-771989239d72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:26.983467 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7bf1e4-f957-4f2e-bd3f-771989239d72 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:27.001057 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 429/843] 10.222.0.40 () {56 vars in 1149 bytes} [Fri Aug 5 10:14:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:27.002312 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e58a4a-fff9-46f4-895f-80baaef810ad None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:27.002597 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e58a4a-fff9-46f4-895f-80baaef810ad None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:27.002871 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e58a4a-fff9-46f4-895f-80baaef810ad None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:27.004241 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 430/844] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:27.006646 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e58a4a-fff9-46f4-895f-80baaef810ad None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:27.018495 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10239b64-774b-4318-a3a4-2974c6aa49f2 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:27.053302 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed755c-7df3-4a13-b64d-fae35222dd26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:27.053854 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed755c-7df3-4a13-b64d-fae35222dd26 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:27.054135 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed755c-7df3-4a13-b64d-fae35222dd26 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:27.054405 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed755c-7df3-4a13-b64d-fae35222dd26 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:27.055175 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed755c-7df3-4a13-b64d-fae35222dd26 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:27.055714 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed755c-7df3-4a13-b64d-fae35222dd26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:27.057394 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed755c-7df3-4a13-b64d-fae35222dd26 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:27.061976 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 415/845] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 5 10:14:26 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:14:27.061976 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76b3df6-44e7-4339-a98f-bb73a53ea21a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:27.061976 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76b3df6-44e7-4339-a98f-bb73a53ea21a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:27.061976 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76b3df6-44e7-4339-a98f-bb73a53ea21a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:27.082959 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a76b3df6-44e7-4339-a98f-bb73a53ea21a None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:27.084152 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a76b3df6-44e7-4339-a98f-bb73a53ea21a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:27.106220 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:27.121739 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e58a4a-fff9-46f4-895f-80baaef810ad None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:27.139679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:27.163249 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f210316e-bdf8-4fa9-bb2a-5b55d4148342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:27.163891 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210316e-bdf8-4fa9-bb2a-5b55d4148342 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:27.164175 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210316e-bdf8-4fa9-bb2a-5b55d4148342 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:27.164442 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210316e-bdf8-4fa9-bb2a-5b55d4148342 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:27.175098 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 416/846] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:27 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:14:27.175098 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76b3df6-44e7-4339-a98f-bb73a53ea21a None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:27.247481 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f210316e-bdf8-4fa9-bb2a-5b55d4148342 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:27.249317 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f210316e-bdf8-4fa9-bb2a-5b55d4148342 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:27.274746 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76b3df6-44e7-4339-a98f-bb73a53ea21a None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:27.311756 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf03273-1a2f-468e-900e-f595a7547920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:27.312302 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf03273-1a2f-468e-900e-f595a7547920 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:27.312726 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf03273-1a2f-468e-900e-f595a7547920 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:27.313007 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf03273-1a2f-468e-900e-f595a7547920 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:27.314190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf03273-1a2f-468e-900e-f595a7547920 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:27.314501 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf03273-1a2f-468e-900e-f595a7547920 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:27.315928 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf03273-1a2f-468e-900e-f595a7547920 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:27.330948 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 417/847] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 5 10:14:27 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:27.330948 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf03273-1a2f-468e-900e-f595a7547920 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:27.353421 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:27.364613 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 431/848] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:14:27 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 05 10:14:27.372068 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f210316e-bdf8-4fa9-bb2a-5b55d4148342 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:27.397131 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:27.476714 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf03273-1a2f-468e-900e-f595a7547920 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:27.493620 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f210316e-bdf8-4fa9-bb2a-5b55d4148342 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:27.517155 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2359fcac-6518-47c9-9395-4b6e0a65c6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:27.518123 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359fcac-6518-47c9-9395-4b6e0a65c6fb None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:27.518582 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359fcac-6518-47c9-9395-4b6e0a65c6fb None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:27.519067 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359fcac-6518-47c9-9395-4b6e0a65c6fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:27.530407 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2359fcac-6518-47c9-9395-4b6e0a65c6fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:14:27.535591 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-2359fcac-6518-47c9-9395-4b6e0a65c6fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:14:27.535591 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:27.535591 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:27.535591 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:27.535591 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:27.535591 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:27.535591 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:27.535591 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:27.535591 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:27.535591 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:27.535591 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:27.535591 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:27.535591 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:14:27.535591 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:14:27.535591 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:14:27.535591 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:14:27.536713 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:14:27.536713 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:14:27.536713 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:14:27.536713 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:14:27.536713 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:27.536713 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:27.536713 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:27.536713 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:27.536713 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:27.536713 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:27.536713 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:27.536713 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:27.536713 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:27.536713 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:27.536713 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:27.536713 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:27.537761 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:27.537761 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:27.537761 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:27.537761 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:27.537761 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:14:27.537761 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:14:27.537761 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:14:27.537761 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:14:27.537761 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:14:27.537761 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:14:27.537761 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:14:27.537761 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:27.537761 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 418/849] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 10:14:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:27.544720 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa486b2-d8fe-4788-b148-4c78669cf3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:27.545210 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2359fcac-6518-47c9-9395-4b6e0a65c6fb None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:27.545783 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa486b2-d8fe-4788-b148-4c78669cf3eb None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:27.546219 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa486b2-d8fe-4788-b148-4c78669cf3eb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:27.546636 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa486b2-d8fe-4788-b148-4c78669cf3eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:27.560733 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa486b2-d8fe-4788-b148-4c78669cf3eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:27.563401 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa486b2-d8fe-4788-b148-4c78669cf3eb None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:27.581445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-6aa486b2-d8fe-4788-b148-4c78669cf3eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:27.581445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:27.581445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:27.581445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:27.581445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:27.581445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:27.581445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:27.581445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:27.581445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:27.581445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:27.581445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:27.581445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:27.581445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:27.581445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:27.581445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:27.582557 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:27.582557 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:27.582557 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:27.582557 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:27.582557 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:27.582557 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:27.582557 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:27.582557 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:27.582557 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:27.582557 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:27.582557 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:27.582557 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:27.582557 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:27.582557 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:27.582557 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:27.582557 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:27.582557 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:27.583567 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:27.583567 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:27.583567 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:27.583567 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:27.584199 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 432/850] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:27.592525 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa486b2-d8fe-4788-b148-4c78669cf3eb None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:27.638627 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:27.669535 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2359fcac-6518-47c9-9395-4b6e0a65c6fb None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:27.678620 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:27.712835 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f0ad5-0696-4fd5-88db-dc56ee0a7619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:27.713419 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f0ad5-0696-4fd5-88db-dc56ee0a7619 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:27.713704 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f0ad5-0696-4fd5-88db-dc56ee0a7619 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:27.713983 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f0ad5-0696-4fd5-88db-dc56ee0a7619 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:27.714805 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9f0ad5-0696-4fd5-88db-dc56ee0a7619 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:27.715172 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9f0ad5-0696-4fd5-88db-dc56ee0a7619 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:27.716662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9f0ad5-0696-4fd5-88db-dc56ee0a7619 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:27.719801 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa486b2-d8fe-4788-b148-4c78669cf3eb None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:27.730177 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 419/851] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:14:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:27.731113 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f0ad5-0696-4fd5-88db-dc56ee0a7619 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:27.757856 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6569b00-4983-469d-8d0c-c4544ba0e45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:27.759001 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6569b00-4983-469d-8d0c-c4544ba0e45c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:27.759685 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6569b00-4983-469d-8d0c-c4544ba0e45c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:27.760201 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6569b00-4983-469d-8d0c-c4544ba0e45c None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:27.761149 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6569b00-4983-469d-8d0c-c4544ba0e45c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:27.761750 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6569b00-4983-469d-8d0c-c4544ba0e45c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:27.763488 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6569b00-4983-469d-8d0c-c4544ba0e45c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:27.783004 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 433/852] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:14:27.784271 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6569b00-4983-469d-8d0c-c4544ba0e45c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:27.833078 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f0ad5-0696-4fd5-88db-dc56ee0a7619 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:27.868221 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e93bc-06e8-430f-b2cc-6283eb7f0da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:27.869250 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e93bc-06e8-430f-b2cc-6283eb7f0da2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:27.869250 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e93bc-06e8-430f-b2cc-6283eb7f0da2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:27.869453 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e93bc-06e8-430f-b2cc-6283eb7f0da2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:27.880877 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797e93bc-06e8-430f-b2cc-6283eb7f0da2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:27.882341 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797e93bc-06e8-430f-b2cc-6283eb7f0da2 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:27.897924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-797e93bc-06e8-430f-b2cc-6283eb7f0da2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:27.897924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:27.897924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:27.897924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:27.897924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:27.897924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:27.897924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:27.897924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:27.897924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:27.897924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:27.897924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:27.897924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:27.897924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:27.897924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:27.897924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:27.898970 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:27.898970 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:27.898970 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:27.898970 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:27.898970 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:27.898970 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:27.898970 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:27.898970 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:27.898970 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:27.898970 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:27.898970 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:27.898970 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:27.898970 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:27.898970 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:27.898970 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:27.898970 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:27.898970 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:27.899887 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:27.899887 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:27.899887 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:27.899887 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:27.900590 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:27.902101 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6569b00-4983-469d-8d0c-c4544ba0e45c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:27.904287 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 420/853] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:14:27.905413 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e93bc-06e8-430f-b2cc-6283eb7f0da2 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:27.935405 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:27.942895 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd5e4e1-9b5d-41a8-a400-b30e93ffedbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:27.944024 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5e4e1-9b5d-41a8-a400-b30e93ffedbc None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:27.944506 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5e4e1-9b5d-41a8-a400-b30e93ffedbc None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:27.944977 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5e4e1-9b5d-41a8-a400-b30e93ffedbc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:27.952712 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd5e4e1-9b5d-41a8-a400-b30e93ffedbc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:27.954431 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd5e4e1-9b5d-41a8-a400-b30e93ffedbc None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:27.971046 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-3fd5e4e1-9b5d-41a8-a400-b30e93ffedbc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:27.971046 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:27.971046 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:27.971046 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:27.971046 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:27.971046 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:27.971046 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:27.971046 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:27.971046 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:27.971046 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:27.971046 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:27.971046 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:27.971046 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 10:14:27.971046 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 10:14:27.971046 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 10:14:27.971992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 10:14:27.971992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:27.971992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:27.971992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:27.971992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:27.971992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:27.971992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:27.971992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:27.971992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:27.971992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:27.971992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:27.971992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:27.971992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:27.971992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:27.971992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:27.971992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:27.971992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:27.972877 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 10:14:27.972877 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 10:14:27.972877 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 10:14:27.972877 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 10:14:27.972877 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 10:14:27.972877 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 10:14:27.972877 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:27.972877 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:27.980033 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 434/854] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:14:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:27.980710 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd5e4e1-9b5d-41a8-a400-b30e93ffedbc None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:28.025223 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e93bc-06e8-430f-b2cc-6283eb7f0da2 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:28.068098 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44d27c4-d3a4-401c-a9f8-6a2e58e15645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:28.069399 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44d27c4-d3a4-401c-a9f8-6a2e58e15645 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:28.069831 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44d27c4-d3a4-401c-a9f8-6a2e58e15645 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:28.070249 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44d27c4-d3a4-401c-a9f8-6a2e58e15645 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:28.071122 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44d27c4-d3a4-401c-a9f8-6a2e58e15645 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:28.071929 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44d27c4-d3a4-401c-a9f8-6a2e58e15645 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:28.073942 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44d27c4-d3a4-401c-a9f8-6a2e58e15645 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:28.089209 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 421/855] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:28.094950 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44d27c4-d3a4-401c-a9f8-6a2e58e15645 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:28.101220 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd5e4e1-9b5d-41a8-a400-b30e93ffedbc None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:28.142190 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a8793-cbd0-4b75-b72a-8c22d525afb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:28.142986 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a8793-cbd0-4b75-b72a-8c22d525afb6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:28.143368 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a8793-cbd0-4b75-b72a-8c22d525afb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:28.143772 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a8793-cbd0-4b75-b72a-8c22d525afb6 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:28.145034 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955a8793-cbd0-4b75-b72a-8c22d525afb6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:28.145448 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955a8793-cbd0-4b75-b72a-8c22d525afb6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:28.147508 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955a8793-cbd0-4b75-b72a-8c22d525afb6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:28.166280 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:28.174385 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 435/856] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 10:14:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:14:28.175830 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a8793-cbd0-4b75-b72a-8c22d525afb6 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:28.225029 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44d27c4-d3a4-401c-a9f8-6a2e58e15645 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:28.241265 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:28.266119 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb85f5-87bb-4de5-99ae-0684c0d37064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:28.266963 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb85f5-87bb-4de5-99ae-0684c0d37064 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:28.267360 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb85f5-87bb-4de5-99ae-0684c0d37064 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:28.267732 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb85f5-87bb-4de5-99ae-0684c0d37064 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:28.277589 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fb85f5-87bb-4de5-99ae-0684c0d37064 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:28.279232 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fb85f5-87bb-4de5-99ae-0684c0d37064 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:28.291482 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-90fb85f5-87bb-4de5-99ae-0684c0d37064 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 05 10:14:28.291482 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:28.291482 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:28.291482 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:28.291482 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:28.291482 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:28.291482 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:28.291482 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:28.291482 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:28.291482 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:28.291482 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:28.291482 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:28.291482 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 10:14:28.291482 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 10:14:28.291482 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 10:14:28.291482 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 10:14:28.293239 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:28.293239 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:28.293239 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 10:14:28.293239 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:28.293239 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 10:14:28.293239 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:28.293239 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:28.293239 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:28.293239 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:28.293239 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:28.293239 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:28.293239 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:28.293239 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:28.293239 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:28.293239 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:28.293239 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:28.293239 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:28.293239 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:28.294451 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:28.294451 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:28.294451 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 10:14:28.294451 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 10:14:28.294451 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 10:14:28.294451 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 10:14:28.294451 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 10:14:28.294451 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 10:14:28.294451 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 05 10:14:28.294451 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:28.300883 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 422/857] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 10:14:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:28.301626 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb85f5-87bb-4de5-99ae-0684c0d37064 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:28.319448 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a8793-cbd0-4b75-b72a-8c22d525afb6 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:28.365415 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc4d03-5a55-4d07-814f-05836fde53c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:28.366937 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc4d03-5a55-4d07-814f-05836fde53c0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:28.367391 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc4d03-5a55-4d07-814f-05836fde53c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:28.368058 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc4d03-5a55-4d07-814f-05836fde53c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:28.369244 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdc4d03-5a55-4d07-814f-05836fde53c0 None admin] RBAC: Adding query filter params (role.id=3a2b4319820a488198dabee34420cdc7, scope.project.id=36b8b505012d4d0bac54845b6799e743, user.id=52fc3ba90dc045c2a5b7bb730da96acc) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:28.369706 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdc4d03-5a55-4d07-814f-05836fde53c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:28.372031 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdc4d03-5a55-4d07-814f-05836fde53c0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:28.387164 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 436/858] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:14:28 2022] GET /identity/v3/role_assignments?user.id=52fc3ba90dc045c2a5b7bb730da96acc&scope.project.id=36b8b505012d4d0bac54845b6799e743&role.id=3a2b4319820a488198dabee34420cdc7 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:14:28.428057 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb85f5-87bb-4de5-99ae-0684c0d37064 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:28.443439 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:28.468436 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1401ece9-3611-4049-a95f-636842774ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:28.469331 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1401ece9-3611-4049-a95f-636842774ef8 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:28.469717 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1401ece9-3611-4049-a95f-636842774ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:28.470106 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1401ece9-3611-4049-a95f-636842774ef8 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:28.471102 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1401ece9-3611-4049-a95f-636842774ef8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:28.471527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1401ece9-3611-4049-a95f-636842774ef8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:28.473171 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1401ece9-3611-4049-a95f-636842774ef8 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:28.495332 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 423/859] 10.222.0.40 () {56 vars in 1135 bytes} [Fri Aug 5 10:14:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:28.501446 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc4d03-5a55-4d07-814f-05836fde53c0 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:28.612898 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc4d03-5a55-4d07-814f-05836fde53c0 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:28.628382 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:28.649284 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c374951-4305-47a6-8fab-2bd26fa9668a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:28.649909 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c374951-4305-47a6-8fab-2bd26fa9668a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:28.650184 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c374951-4305-47a6-8fab-2bd26fa9668a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:28.650454 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c374951-4305-47a6-8fab-2bd26fa9668a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:28.723570 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c374951-4305-47a6-8fab-2bd26fa9668a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:28.725034 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c374951-4305-47a6-8fab-2bd26fa9668a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:28.796795 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3986109-1eaf-4d76-be15-74f85eae2646 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:28.797154 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3986109-1eaf-4d76-be15-74f85eae2646 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:28.797357 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3986109-1eaf-4d76-be15-74f85eae2646 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:28.798843 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 424/860] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:28.804104 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2afa9a-e834-427d-a07b-7adc6e99dfba None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:28.804663 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2afa9a-e834-427d-a07b-7adc6e99dfba None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:28.805197 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2afa9a-e834-427d-a07b-7adc6e99dfba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:28.824176 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-9b2afa9a-e834-427d-a07b-7adc6e99dfba None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:28.825723 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9b2afa9a-e834-427d-a07b-7adc6e99dfba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:28.832621 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 437/861] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:14:28 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 10:14:28.838322 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c374951-4305-47a6-8fab-2bd26fa9668a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:28.853349 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:28.901643 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 425/862] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:28 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 10:14:28.927241 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2afa9a-e834-427d-a07b-7adc6e99dfba None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:28.944583 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c374951-4305-47a6-8fab-2bd26fa9668a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:28.979971 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb70e50-ada4-4079-b92c-f32314e1f79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:28.980683 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb70e50-ada4-4079-b92c-f32314e1f79c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:28.980971 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb70e50-ada4-4079-b92c-f32314e1f79c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:28.981248 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb70e50-ada4-4079-b92c-f32314e1f79c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:28.992026 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb70e50-ada4-4079-b92c-f32314e1f79c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:28.993513 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb70e50-ada4-4079-b92c-f32314e1f79c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:29.009090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-fdb70e50-ada4-4079-b92c-f32314e1f79c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:29.009090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:29.009090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:29.009090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:29.009090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:29.009090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:29.009090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:29.009090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:29.009090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:29.009090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:29.009090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:29.009090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:29.009090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:29.009090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:29.009090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:29.010068 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:29.010068 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:29.010068 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:29.010068 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:29.010068 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:29.010068 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:29.010068 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:29.010068 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:29.010068 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:29.010068 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:29.010068 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:29.010068 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:29.010068 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:29.010068 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:29.010068 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:29.010068 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:29.010068 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:29.011037 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:29.011037 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:29.011037 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:29.011037 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:29.011037 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 438/863] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:14:29.016453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb70e50-ada4-4079-b92c-f32314e1f79c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:29.036811 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:29.040447 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2afa9a-e834-427d-a07b-7adc6e99dfba None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:29.079086 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e40ca1-50f1-4a2b-ba08-7c5e7b9674ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:29.079782 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e40ca1-50f1-4a2b-ba08-7c5e7b9674ec None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:29.080065 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e40ca1-50f1-4a2b-ba08-7c5e7b9674ec None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:29.080340 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e40ca1-50f1-4a2b-ba08-7c5e7b9674ec None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:29.080970 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e40ca1-50f1-4a2b-ba08-7c5e7b9674ec None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:29.082353 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e40ca1-50f1-4a2b-ba08-7c5e7b9674ec None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:29.086975 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:29.095998 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-96e40ca1-50f1-4a2b-ba08-7c5e7b9674ec None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 10:14:29.095998 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:29.095998 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 10:14:29.095998 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 10:14:29.095998 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 10:14:29.095998 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 10:14:29.095998 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 10:14:29.095998 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:29.095998 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 10:14:29.095998 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:29.095998 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 10:14:29.095998 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:29.095998 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:29.095998 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:29.095998 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:29.095998 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:29.096978 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:29.096978 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:29.096978 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:29.096978 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:29.096978 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:29.096978 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:29.096978 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:29.096978 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 10:14:29.096978 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 10:14:29.096978 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 10:14:29.096978 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 10:14:29.096978 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:29.096978 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:29.096978 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:29.096978 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:29.096978 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:29.097810 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:29.097810 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:29.097810 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:29.097810 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:29.097810 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:29.097810 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:29.097810 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:29.097810 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:29.097810 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:29.097810 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:29.097810 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:29.097810 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 10:14:29.097810 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:29.097810 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 10:14:29.097810 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:29.102464 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 426/864] 10.222.0.40 () {56 vars in 1132 bytes} [Fri Aug 5 10:14:28 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:14:29.103346 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e40ca1-50f1-4a2b-ba08-7c5e7b9674ec None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:29.142238 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb70e50-ada4-4079-b92c-f32314e1f79c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:29.195469 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e30a7b-ad0d-4f27-a0c9-86a421591c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:29.196742 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e30a7b-ad0d-4f27-a0c9-86a421591c1f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:29.197700 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e30a7b-ad0d-4f27-a0c9-86a421591c1f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:29.198183 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e30a7b-ad0d-4f27-a0c9-86a421591c1f None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:29.199544 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e30a7b-ad0d-4f27-a0c9-86a421591c1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:29.200305 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e30a7b-ad0d-4f27-a0c9-86a421591c1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:29.202308 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e30a7b-ad0d-4f27-a0c9-86a421591c1f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:29.216535 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e40ca1-50f1-4a2b-ba08-7c5e7b9674ec None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:29.225326 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 439/865] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:14:29.226502 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e30a7b-ad0d-4f27-a0c9-86a421591c1f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:29.254938 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7aa69d-8838-4c3f-abb3-3efd1d00d0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:29.255557 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7aa69d-8838-4c3f-abb3-3efd1d00d0b4 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:29.255839 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7aa69d-8838-4c3f-abb3-3efd1d00d0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:29.256112 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7aa69d-8838-4c3f-abb3-3efd1d00d0b4 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:29.256854 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7aa69d-8838-4c3f-abb3-3efd1d00d0b4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:29.257159 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7aa69d-8838-4c3f-abb3-3efd1d00d0b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:29.258629 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7aa69d-8838-4c3f-abb3-3efd1d00d0b4 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:29.263837 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 427/866] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Aug 5 10:14:29 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:29.269521 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7aa69d-8838-4c3f-abb3-3efd1d00d0b4 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:29.301795 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:29.364802 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e30a7b-ad0d-4f27-a0c9-86a421591c1f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:29.379445 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7aa69d-8838-4c3f-abb3-3efd1d00d0b4 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:29.400500 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:29.410114 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a33360-32b5-40bb-9222-3ca25b34be51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:29.410704 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a33360-32b5-40bb-9222-3ca25b34be51 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:29.411017 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a33360-32b5-40bb-9222-3ca25b34be51 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:29.411288 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a33360-32b5-40bb-9222-3ca25b34be51 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:29.414996 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25a949-e8a2-4a61-88a6-7a0d58e71174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:29.415616 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25a949-e8a2-4a61-88a6-7a0d58e71174 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:29.415894 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25a949-e8a2-4a61-88a6-7a0d58e71174 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:29.416158 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25a949-e8a2-4a61-88a6-7a0d58e71174 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:29.416830 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25a949-e8a2-4a61-88a6-7a0d58e71174 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:29.417250 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25a949-e8a2-4a61-88a6-7a0d58e71174 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:29.418757 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a33360-32b5-40bb-9222-3ca25b34be51 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:29.419130 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25a949-e8a2-4a61-88a6-7a0d58e71174 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:29.420236 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a33360-32b5-40bb-9222-3ca25b34be51 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:29.430633 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 428/867] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Aug 5 10:14:29 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:29.430633 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca94c2e6-c418-4d93-ab97-e2cb6e1efc6d None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:29.430633 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca94c2e6-c418-4d93-ab97-e2cb6e1efc6d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:29.430633 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca94c2e6-c418-4d93-ab97-e2cb6e1efc6d None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:29.430633 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 429/868] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:29.432330 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b8791-aee8-4f60-9501-03f1f37f8228 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:29.432616 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b8791-aee8-4f60-9501-03f1f37f8228 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:29.433075 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b8791-aee8-4f60-9501-03f1f37f8228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:29.438084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-99a33360-32b5-40bb-9222-3ca25b34be51 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:29.438084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:29.438084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:29.438084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:29.438084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:29.438084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:29.438084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:29.438084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:29.438084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:29.438084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:29.438084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:29.438084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:29.438084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 10:14:29.438084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 10:14:29.438084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 10:14:29.439054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 10:14:29.439054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:29.439054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:29.439054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:29.439054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:29.439054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:29.439054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:29.439054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:29.439054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:29.439054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:29.439054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:29.439054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:29.439054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:29.439054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:29.439054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:29.439054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:29.439054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:29.439967 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 10:14:29.439967 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 10:14:29.439967 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 10:14:29.439967 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 10:14:29.439967 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 10:14:29.439967 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 10:14:29.439967 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:29.439967 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:29.439967 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 440/869] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:14:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:29.447418 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a33360-32b5-40bb-9222-3ca25b34be51 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:29.453733 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b32b8791-aee8-4f60-9501-03f1f37f8228 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:29.455141 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b32b8791-aee8-4f60-9501-03f1f37f8228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:29.547106 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 430/870] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:29 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 10:14:29.567688 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a33360-32b5-40bb-9222-3ca25b34be51 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:29.572645 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b8791-aee8-4f60-9501-03f1f37f8228 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:29.605257 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f56593-d076-4f1e-ac7b-61f318118230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:29.605907 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f56593-d076-4f1e-ac7b-61f318118230 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:29.606192 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f56593-d076-4f1e-ac7b-61f318118230 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:29.606468 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f56593-d076-4f1e-ac7b-61f318118230 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:29.608178 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f56593-d076-4f1e-ac7b-61f318118230 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:29.608178 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f56593-d076-4f1e-ac7b-61f318118230 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:29.610152 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f56593-d076-4f1e-ac7b-61f318118230 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:29.612805 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:29.623490 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 441/871] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 10:14:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:14:29.631956 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f56593-d076-4f1e-ac7b-61f318118230 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:29.691986 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:29.711701 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b8791-aee8-4f60-9501-03f1f37f8228 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:29.753278 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f56593-d076-4f1e-ac7b-61f318118230 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:29.758547 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2356c7f9-cd3d-4358-85b5-2ec3c7786151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:29.759211 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2356c7f9-cd3d-4358-85b5-2ec3c7786151 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:29.759491 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2356c7f9-cd3d-4358-85b5-2ec3c7786151 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:29.759762 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2356c7f9-cd3d-4358-85b5-2ec3c7786151 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:29.760458 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2356c7f9-cd3d-4358-85b5-2ec3c7786151 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:29.762523 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2356c7f9-cd3d-4358-85b5-2ec3c7786151 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:29.791115 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 431/872] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 5 10:14:29 2022] POST /identity/v3/services => generated 251 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:14:29.797420 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb97c07c-dba8-47a6-8e26-4b33ec94ee90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:29.798127 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb97c07c-dba8-47a6-8e26-4b33ec94ee90 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:29.798416 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb97c07c-dba8-47a6-8e26-4b33ec94ee90 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:29.798688 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb97c07c-dba8-47a6-8e26-4b33ec94ee90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:29.799537 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb97c07c-dba8-47a6-8e26-4b33ec94ee90 None admin] RBAC: Adding query filter params (role.id=daef89adcaeb48a382a0bcb03340ccbb, scope.project.id=36b8b505012d4d0bac54845b6799e743, user.id=c80cf0e429f04f229aea1aa9293a93b2) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:29.799849 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb97c07c-dba8-47a6-8e26-4b33ec94ee90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:29.801384 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb97c07c-dba8-47a6-8e26-4b33ec94ee90 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:29.809583 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 442/873] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:14:29 2022] GET /identity/v3/role_assignments?user.id=c80cf0e429f04f229aea1aa9293a93b2&scope.project.id=36b8b505012d4d0bac54845b6799e743&role.id=daef89adcaeb48a382a0bcb03340ccbb => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:30.553449 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d305dd-28fe-4cbc-8854-c766c620518f None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:30.554094 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d305dd-28fe-4cbc-8854-c766c620518f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:30.554094 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d305dd-28fe-4cbc-8854-c766c620518f None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:30.555679 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 432/874] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:30.561038 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d6190c-ac8a-4f78-8991-59508199c47d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:30.561461 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d6190c-ac8a-4f78-8991-59508199c47d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:30.561605 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d6190c-ac8a-4f78-8991-59508199c47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:30.579810 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-09d6190c-ac8a-4f78-8991-59508199c47d None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:30.580985 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-09d6190c-ac8a-4f78-8991-59508199c47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:30.652671 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 443/875] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:30 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:14:30.675443 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d305dd-28fe-4cbc-8854-c766c620518f None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:30.759096 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:30.779529 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d305dd-28fe-4cbc-8854-c766c620518f None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:30.812836 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c27ea72-8024-4339-a704-d8fdf451dd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:30.813459 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c27ea72-8024-4339-a704-d8fdf451dd9b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:30.813742 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c27ea72-8024-4339-a704-d8fdf451dd9b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:30.814011 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c27ea72-8024-4339-a704-d8fdf451dd9b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:30.824983 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c27ea72-8024-4339-a704-d8fdf451dd9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:14:30.829947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-2c27ea72-8024-4339-a704-d8fdf451dd9b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 10:14:30.829947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:30.829947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:30.829947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:30.829947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:30.829947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:30.829947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:30.829947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:30.829947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:30.829947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:30.829947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:30.829947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:30.829947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:14:30.829947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:14:30.829947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:14:30.829947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:14:30.830853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:14:30.830853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:14:30.830853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:14:30.830853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:14:30.830853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:30.830853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:30.830853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:30.830853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:30.830853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:30.830853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:30.830853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:30.830853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:30.830853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:30.830853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:30.830853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:30.830853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:30.831723 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:30.831723 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:30.831723 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:30.831723 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:30.831723 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:14:30.831723 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:14:30.831723 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:14:30.831723 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:14:30.831723 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:14:30.831723 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:14:30.831723 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 10:14:30.831723 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:30.835038 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 433/876] 10.222.0.40 () {56 vars in 1120 bytes} [Fri Aug 5 10:14:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:30.837829 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d6190c-ac8a-4f78-8991-59508199c47d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:30.875229 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:30.943090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d6190c-ac8a-4f78-8991-59508199c47d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:30.979147 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12037ab-9c61-4df5-9af7-d463a8563248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:30.979880 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12037ab-9c61-4df5-9af7-d463a8563248 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:30.980156 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12037ab-9c61-4df5-9af7-d463a8563248 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:30.980502 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12037ab-9c61-4df5-9af7-d463a8563248 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:30.981264 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12037ab-9c61-4df5-9af7-d463a8563248 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:30.981575 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12037ab-9c61-4df5-9af7-d463a8563248 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:30.983050 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12037ab-9c61-4df5-9af7-d463a8563248 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:30.993584 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 444/877] 10.222.0.40 () {56 vars in 1115 bytes} [Fri Aug 5 10:14:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:30.999491 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c27ea72-8024-4339-a704-d8fdf451dd9b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:31.104711 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c27ea72-8024-4339-a704-d8fdf451dd9b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:31.138595 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb9da1a-4288-411c-a06b-b4bf8c137eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:31.139268 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9da1a-4288-411c-a06b-b4bf8c137eb0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:31.139542 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9da1a-4288-411c-a06b-b4bf8c137eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:31.139809 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9da1a-4288-411c-a06b-b4bf8c137eb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:31.150522 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb9da1a-4288-411c-a06b-b4bf8c137eb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:31.153952 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb9da1a-4288-411c-a06b-b4bf8c137eb0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:31.170925 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-bfb9da1a-4288-411c-a06b-b4bf8c137eb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:31.170925 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:31.170925 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:31.170925 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:31.170925 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:31.170925 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:31.170925 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:31.170925 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:31.170925 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:31.170925 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:31.170925 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:31.170925 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:31.170925 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:31.170925 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:31.170925 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:31.171837 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:31.171837 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:31.171837 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:31.171837 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:31.171837 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:31.171837 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:31.171837 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:31.171837 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:31.171837 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:31.171837 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:31.171837 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:31.171837 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:31.171837 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:31.171837 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:31.171837 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:31.171837 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:31.171837 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:31.172807 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:31.172807 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:31.172807 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:31.172807 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:31.172807 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 434/878] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:31.172807 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:31.176051 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12037ab-9c61-4df5-9af7-d463a8563248 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:31.283022 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:31.287542 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12037ab-9c61-4df5-9af7-d463a8563248 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:31.323638 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756cfc0-9c87-4605-938b-8d63697bb969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:31.324625 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756cfc0-9c87-4605-938b-8d63697bb969 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:31.325106 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756cfc0-9c87-4605-938b-8d63697bb969 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:31.325580 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756cfc0-9c87-4605-938b-8d63697bb969 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:31.326498 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2756cfc0-9c87-4605-938b-8d63697bb969 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:31.327035 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2756cfc0-9c87-4605-938b-8d63697bb969 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:31.328607 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2756cfc0-9c87-4605-938b-8d63697bb969 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:31.341909 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 445/879] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:31.347331 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb9da1a-4288-411c-a06b-b4bf8c137eb0 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:31.447610 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb9da1a-4288-411c-a06b-b4bf8c137eb0 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:31.484650 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6942ac90-4929-48f1-ad4f-758ae1d0db5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:31.485299 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942ac90-4929-48f1-ad4f-758ae1d0db5b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:31.485575 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942ac90-4929-48f1-ad4f-758ae1d0db5b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:31.485845 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942ac90-4929-48f1-ad4f-758ae1d0db5b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:31.493794 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6942ac90-4929-48f1-ad4f-758ae1d0db5b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:31.495396 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6942ac90-4929-48f1-ad4f-758ae1d0db5b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:31.507048 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-6942ac90-4929-48f1-ad4f-758ae1d0db5b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 10:14:31.507048 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:31.507048 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:31.507048 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:31.507048 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:31.507048 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:31.507048 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:31.507048 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:31.507048 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:31.507048 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:31.507048 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:31.507048 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:31.507048 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 10:14:31.507048 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 10:14:31.507048 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 10:14:31.507048 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 10:14:31.508039 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:31.508039 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:31.508039 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 10:14:31.508039 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:31.508039 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 10:14:31.508039 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:31.508039 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:31.508039 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:31.508039 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:31.508039 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:31.508039 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:31.508039 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:31.508039 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:31.508039 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:31.508039 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:31.508039 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:31.508039 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:31.508039 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:31.508964 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:31.508964 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:31.508964 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 10:14:31.508964 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 10:14:31.508964 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 10:14:31.508964 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 10:14:31.508964 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 10:14:31.508964 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 10:14:31.508964 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 10:14:31.508964 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:31.508964 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 435/880] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Aug 5 10:14:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:31.514728 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756cfc0-9c87-4605-938b-8d63697bb969 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:31.571378 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6870204-7b71-4406-927b-57884f838ec9 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:31.571904 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6870204-7b71-4406-927b-57884f838ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:31.572008 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6870204-7b71-4406-927b-57884f838ec9 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:31.573689 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 436/881] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:31.580231 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a5c69-dfed-4ff7-984c-fa1c3176e7a8 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:31.580515 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a5c69-dfed-4ff7-984c-fa1c3176e7a8 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:31.580847 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a5c69-dfed-4ff7-984c-fa1c3176e7a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:31.599055 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c29a5c69-dfed-4ff7-984c-fa1c3176e7a8 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:31.600232 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c29a5c69-dfed-4ff7-984c-fa1c3176e7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:31.629284 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756cfc0-9c87-4605-938b-8d63697bb969 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:31.653611 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:31.672033 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a444f39c-dcc1-425b-b98e-21c829241ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:31.672719 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444f39c-dcc1-425b-b98e-21c829241ae2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:31.672998 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444f39c-dcc1-425b-b98e-21c829241ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:31.673268 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444f39c-dcc1-425b-b98e-21c829241ae2 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:31.674063 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a444f39c-dcc1-425b-b98e-21c829241ae2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:31.674366 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a444f39c-dcc1-425b-b98e-21c829241ae2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:31.674954 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 437/882] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:31 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 10:14:31.679925 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a444f39c-dcc1-425b-b98e-21c829241ae2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:31.697556 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29a5c69-dfed-4ff7-984c-fa1c3176e7a8 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:31.699158 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 446/883] 10.222.0.40 () {56 vars in 1149 bytes} [Fri Aug 5 10:14:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:31.704731 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a444f39c-dcc1-425b-b98e-21c829241ae2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:31.722050 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:31.808008 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a444f39c-dcc1-425b-b98e-21c829241ae2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:31.812001 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29a5c69-dfed-4ff7-984c-fa1c3176e7a8 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:31.842752 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89f9693-8bbb-4820-8a99-b749b3843a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:31.843781 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89f9693-8bbb-4820-8a99-b749b3843a99 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:31.844211 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89f9693-8bbb-4820-8a99-b749b3843a99 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:31.844664 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89f9693-8bbb-4820-8a99-b749b3843a99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:31.849866 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b77e38-5e4a-4ebc-b170-4d486b6b1f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:31.850604 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b77e38-5e4a-4ebc-b170-4d486b6b1f45 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:31.851016 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b77e38-5e4a-4ebc-b170-4d486b6b1f45 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:31.851389 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b77e38-5e4a-4ebc-b170-4d486b6b1f45 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:31.852188 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b77e38-5e4a-4ebc-b170-4d486b6b1f45 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:31.853669 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b77e38-5e4a-4ebc-b170-4d486b6b1f45 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:31.872598 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 438/884] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 5 10:14:31 2022] POST /identity/v3/services => generated 269 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:14:31.894882 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf903ac-6e2a-4f35-8a2e-a606707954b2 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:31.895208 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf903ac-6e2a-4f35-8a2e-a606707954b2 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:31.895489 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf903ac-6e2a-4f35-8a2e-a606707954b2 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:31.897205 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 439/885] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 10:14:31.903166 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e198c-b3ed-4fe6-b5fa-c26521a5ae3a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:31.903451 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e198c-b3ed-4fe6-b5fa-c26521a5ae3a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:31.903736 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e198c-b3ed-4fe6-b5fa-c26521a5ae3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:31.926544 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-d52e198c-b3ed-4fe6-b5fa-c26521a5ae3a None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:31.928162 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d52e198c-b3ed-4fe6-b5fa-c26521a5ae3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:31.928773 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89f9693-8bbb-4820-8a99-b749b3843a99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:31.930605 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89f9693-8bbb-4820-8a99-b749b3843a99 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:31.951262 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:31.990429 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:32.031259 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 440/886] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:31 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 10:14:32.032702 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e978b-a1e8-46a4-8547-71eb68b08b4e None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:32.033061 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e978b-a1e8-46a4-8547-71eb68b08b4e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:32.033406 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e978b-a1e8-46a4-8547-71eb68b08b4e None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:32.034700 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 441/887] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:32.039971 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2888f6-7d01-4fad-8a82-24fff0b2cc7d None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:32.040258 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2888f6-7d01-4fad-8a82-24fff0b2cc7d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:32.040629 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2888f6-7d01-4fad-8a82-24fff0b2cc7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:32.048373 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 447/888] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:14:31 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 05 10:14:32.054081 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89f9693-8bbb-4820-8a99-b749b3843a99 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:32.059592 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ee2888f6-7d01-4fad-8a82-24fff0b2cc7d None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:32.060794 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ee2888f6-7d01-4fad-8a82-24fff0b2cc7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:32.129891 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 442/889] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:32 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 10:14:32.131150 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2888f6-7d01-4fad-8a82-24fff0b2cc7d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:32.163160 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89f9693-8bbb-4820-8a99-b749b3843a99 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:32.181513 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:32.205167 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9398fc-3615-4669-9c8b-b154a539bc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:32.205860 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9398fc-3615-4669-9c8b-b154a539bc49 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:32.206140 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9398fc-3615-4669-9c8b-b154a539bc49 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:32.206414 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9398fc-3615-4669-9c8b-b154a539bc49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:32.217523 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9398fc-3615-4669-9c8b-b154a539bc49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:32.219127 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9398fc-3615-4669-9c8b-b154a539bc49 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:32.235510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-3d9398fc-3615-4669-9c8b-b154a539bc49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:32.235510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:32.235510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:32.235510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:32.235510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:32.235510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:32.235510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:32.235510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:32.235510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:32.235510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:32.235510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:32.235510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:32.235510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:32.235510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:32.235510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:32.236461 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:32.236461 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:32.236461 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:32.236461 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:32.236461 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:32.236461 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:32.236461 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:32.236461 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:32.236461 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:32.236461 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:32.236461 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:32.236461 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:32.236461 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:32.236461 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:32.236461 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:32.236461 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:32.236461 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:32.237361 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:32.237361 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:32.237361 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:32.237361 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:32.237361 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 448/890] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:32.237642 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9398fc-3615-4669-9c8b-b154a539bc49 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:32.244359 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:32.248009 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2888f6-7d01-4fad-8a82-24fff0b2cc7d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:32.287356 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0e83d-b6c5-4b01-9925-2d9b42caeb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:32.288005 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0e83d-b6c5-4b01-9925-2d9b42caeb6b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:32.288287 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0e83d-b6c5-4b01-9925-2d9b42caeb6b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:32.288557 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0e83d-b6c5-4b01-9925-2d9b42caeb6b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:32.289306 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca0e83d-b6c5-4b01-9925-2d9b42caeb6b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:32.290868 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca0e83d-b6c5-4b01-9925-2d9b42caeb6b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:32.300861 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-bca0e83d-b6c5-4b01-9925-2d9b42caeb6b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 10:14:32.300861 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:32.300861 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 10:14:32.300861 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 10:14:32.300861 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 10:14:32.300861 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 10:14:32.300861 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 10:14:32.300861 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:32.300861 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 10:14:32.300861 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:32.300861 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 10:14:32.300861 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:32.300861 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:32.300861 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:32.300861 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:32.300861 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:32.300861 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:32.301848 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:32.301848 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:32.301848 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:32.301848 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:32.301848 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:32.301848 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:32.301848 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 10:14:32.301848 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 10:14:32.301848 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 10:14:32.301848 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 10:14:32.301848 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:32.301848 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:32.301848 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:32.301848 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:32.301848 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:32.301848 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:32.302674 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:32.302674 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:32.302674 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:32.302674 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:32.302674 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:32.302674 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:32.302674 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:32.302674 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:32.302674 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:32.302674 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:32.302674 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 10:14:32.302674 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:32.302674 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 10:14:32.302674 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:32.302674 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 443/891] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Aug 5 10:14:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:32.303516 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0e83d-b6c5-4b01-9925-2d9b42caeb6b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:32.346066 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9398fc-3615-4669-9c8b-b154a539bc49 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:32.381212 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5b827-cef0-4233-9bb3-fb793a0b5d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:32.381852 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5b827-cef0-4233-9bb3-fb793a0b5d51 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:32.382127 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5b827-cef0-4233-9bb3-fb793a0b5d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:32.382387 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5b827-cef0-4233-9bb3-fb793a0b5d51 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:32.383053 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c5b827-cef0-4233-9bb3-fb793a0b5d51 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:32.384421 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c5b827-cef0-4233-9bb3-fb793a0b5d51 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:32.393453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-54c5b827-cef0-4233-9bb3-fb793a0b5d51 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 10:14:32.393453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:32.393453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 10:14:32.393453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 10:14:32.393453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 10:14:32.393453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 10:14:32.393453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 10:14:32.393453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:32.393453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 10:14:32.393453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:32.393453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 10:14:32.393453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:32.393453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:32.393453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:32.393453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:32.393453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:32.393453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:32.394456 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:32.394456 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:32.394456 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:32.394456 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:32.394456 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:32.394456 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:32.394456 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 10:14:32.394456 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 10:14:32.394456 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 10:14:32.394456 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 10:14:32.394456 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:32.394456 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:32.394456 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:32.394456 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:32.394456 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:32.394456 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:32.395350 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:32.395350 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:32.395350 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:32.395350 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:32.395350 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:32.395350 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:32.395350 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:32.395350 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:32.395350 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:32.395350 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:32.395350 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 10:14:32.395350 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:32.395350 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 10:14:32.395350 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:32.395350 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 449/892] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 5 10:14:32 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:32.396159 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5b827-cef0-4233-9bb3-fb793a0b5d51 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:32.425477 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0e83d-b6c5-4b01-9925-2d9b42caeb6b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:32.431799 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:32.461596 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a385cf-5606-40d0-a95f-e9c5f473aaff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:32.462221 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a385cf-5606-40d0-a95f-e9c5f473aaff None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:32.462497 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a385cf-5606-40d0-a95f-e9c5f473aaff None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:32.462763 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a385cf-5606-40d0-a95f-e9c5f473aaff None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:32.463483 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a385cf-5606-40d0-a95f-e9c5f473aaff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:32.463824 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a385cf-5606-40d0-a95f-e9c5f473aaff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:32.465346 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a385cf-5606-40d0-a95f-e9c5f473aaff None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:32.478552 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 444/893] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:32.479741 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a385cf-5606-40d0-a95f-e9c5f473aaff None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:32.506533 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:32.506851 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5b827-cef0-4233-9bb3-fb793a0b5d51 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:32.548161 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e8b0f3-0b9f-466a-9376-10ae6fabab0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:32.548792 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e8b0f3-0b9f-466a-9376-10ae6fabab0e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:32.549075 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e8b0f3-0b9f-466a-9376-10ae6fabab0e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:32.549596 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e8b0f3-0b9f-466a-9376-10ae6fabab0e None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:32.550312 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e8b0f3-0b9f-466a-9376-10ae6fabab0e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:32.551195 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e8b0f3-0b9f-466a-9376-10ae6fabab0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:32.552958 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e8b0f3-0b9f-466a-9376-10ae6fabab0e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:32.564214 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 450/894] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Aug 5 10:14:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:32.564214 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e8b0f3-0b9f-466a-9376-10ae6fabab0e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:32.586408 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a385cf-5606-40d0-a95f-e9c5f473aaff None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:32.622561 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9636ef-123d-400d-ad44-ca0dc142a664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:32.623501 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9636ef-123d-400d-ad44-ca0dc142a664 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:32.623928 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9636ef-123d-400d-ad44-ca0dc142a664 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:32.624360 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9636ef-123d-400d-ad44-ca0dc142a664 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:32.625219 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9636ef-123d-400d-ad44-ca0dc142a664 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:32.625670 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9636ef-123d-400d-ad44-ca0dc142a664 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:32.627228 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9636ef-123d-400d-ad44-ca0dc142a664 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:32.632112 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 445/895] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 5 10:14:32 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:32.633058 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9636ef-123d-400d-ad44-ca0dc142a664 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:32.670204 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:32.676264 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e8b0f3-0b9f-466a-9376-10ae6fabab0e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:32.710855 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92445622-6ec6-4cb0-8e79-2cd7ea5de35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:32.711913 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92445622-6ec6-4cb0-8e79-2cd7ea5de35e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:32.712134 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92445622-6ec6-4cb0-8e79-2cd7ea5de35e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:32.712699 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92445622-6ec6-4cb0-8e79-2cd7ea5de35e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:32.720146 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92445622-6ec6-4cb0-8e79-2cd7ea5de35e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:32.721573 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92445622-6ec6-4cb0-8e79-2cd7ea5de35e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:32.722368 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:32.733459 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-92445622-6ec6-4cb0-8e79-2cd7ea5de35e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:32.733459 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:32.733459 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:32.733459 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:32.733459 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:32.733459 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:32.733459 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:32.733459 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:32.733459 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:32.733459 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:32.733459 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:32.733459 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:32.733459 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 10:14:32.733459 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 10:14:32.733459 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 10:14:32.734437 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 10:14:32.734437 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:32.734437 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:32.734437 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:32.734437 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:32.734437 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:32.734437 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:32.734437 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:32.734437 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:32.734437 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:32.734437 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:32.734437 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:32.734437 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:32.734437 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:32.734437 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:32.734437 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:32.734437 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:32.740577 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 10:14:32.740577 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 10:14:32.740577 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 10:14:32.740577 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 10:14:32.740577 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 10:14:32.740577 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 10:14:32.740577 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:32.740577 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:32.741003 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9636ef-123d-400d-ad44-ca0dc142a664 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:32.741086 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 451/896] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:14:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:32.741086 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92445622-6ec6-4cb0-8e79-2cd7ea5de35e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:32.774213 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453da9b-9fcd-4ec2-9730-dd1cb1676b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:32.774872 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453da9b-9fcd-4ec2-9730-dd1cb1676b3b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:32.775186 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453da9b-9fcd-4ec2-9730-dd1cb1676b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:32.775468 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453da9b-9fcd-4ec2-9730-dd1cb1676b3b None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:32.776341 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7453da9b-9fcd-4ec2-9730-dd1cb1676b3b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:32.776662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7453da9b-9fcd-4ec2-9730-dd1cb1676b3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:32.777983 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7453da9b-9fcd-4ec2-9730-dd1cb1676b3b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:32.786987 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 446/897] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Aug 5 10:14:32 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:32.786987 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453da9b-9fcd-4ec2-9730-dd1cb1676b3b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:32.875915 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92445622-6ec6-4cb0-8e79-2cd7ea5de35e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:32.899067 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453da9b-9fcd-4ec2-9730-dd1cb1676b3b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:32.920295 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8092e64a-5d22-4993-a0ee-41341ee2160e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:32.921043 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092e64a-5d22-4993-a0ee-41341ee2160e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:32.921869 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092e64a-5d22-4993-a0ee-41341ee2160e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:32.922206 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092e64a-5d22-4993-a0ee-41341ee2160e None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:32.923264 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8092e64a-5d22-4993-a0ee-41341ee2160e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:32.923634 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8092e64a-5d22-4993-a0ee-41341ee2160e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:32.925640 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8092e64a-5d22-4993-a0ee-41341ee2160e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:32.933475 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 452/898] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 5 10:14:32 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:32.938132 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e34b4-eaea-4b30-9375-20c22593ebb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:32.938787 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e34b4-eaea-4b30-9375-20c22593ebb2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:32.939125 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e34b4-eaea-4b30-9375-20c22593ebb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:32.939246 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8092e64a-5d22-4993-a0ee-41341ee2160e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:32.939408 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e34b4-eaea-4b30-9375-20c22593ebb2 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:32.940216 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239e34b4-eaea-4b30-9375-20c22593ebb2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:32.940523 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239e34b4-eaea-4b30-9375-20c22593ebb2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:32.942145 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239e34b4-eaea-4b30-9375-20c22593ebb2 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:32.960836 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 447/899] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 10:14:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:32.961865 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e34b4-eaea-4b30-9375-20c22593ebb2 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:32.969696 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:32.980359 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:33.053479 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8092e64a-5d22-4993-a0ee-41341ee2160e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:33.087887 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588eae66-b006-4a55-bebe-9e2257cf2e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:33.088824 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588eae66-b006-4a55-bebe-9e2257cf2e01 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:33.089108 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588eae66-b006-4a55-bebe-9e2257cf2e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:33.089535 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588eae66-b006-4a55-bebe-9e2257cf2e01 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:33.090615 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588eae66-b006-4a55-bebe-9e2257cf2e01 None admin] RBAC: Adding query filter params (interface=public, service_id=ca9fe4074a614f1797be2a067add44a6, region_id=RegionOne) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:33.091085 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588eae66-b006-4a55-bebe-9e2257cf2e01 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:33.092612 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588eae66-b006-4a55-bebe-9e2257cf2e01 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:33.093007 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e34b4-eaea-4b30-9375-20c22593ebb2 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:33.099149 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 453/900] 10.222.0.40 () {56 vars in 1251 bytes} [Fri Aug 5 10:14:32 2022] GET /identity/v3/endpoints?service_id=ca9fe4074a614f1797be2a067add44a6&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:14:33.112658 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588eae66-b006-4a55-bebe-9e2257cf2e01 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:33.135889 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864f0e7-d184-48a0-8ac5-d33b7ddd3569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:33.136881 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864f0e7-d184-48a0-8ac5-d33b7ddd3569 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:33.137170 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864f0e7-d184-48a0-8ac5-d33b7ddd3569 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:33.137442 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864f0e7-d184-48a0-8ac5-d33b7ddd3569 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:33.138287 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9864f0e7-d184-48a0-8ac5-d33b7ddd3569 None admin] RBAC: Adding query filter params (role.id=f10badb1efd340eda8d6358dc6bd9c28, scope.project.id=36b8b505012d4d0bac54845b6799e743, user.id=52fc3ba90dc045c2a5b7bb730da96acc) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:33.138647 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9864f0e7-d184-48a0-8ac5-d33b7ddd3569 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:33.140916 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9864f0e7-d184-48a0-8ac5-d33b7ddd3569 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:33.148982 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 448/901] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:14:32 2022] GET /identity/v3/role_assignments?user.id=52fc3ba90dc045c2a5b7bb730da96acc&scope.project.id=36b8b505012d4d0bac54845b6799e743&role.id=f10badb1efd340eda8d6358dc6bd9c28 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:33.226205 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:33.246192 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588eae66-b006-4a55-bebe-9e2257cf2e01 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:33.284956 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400b9b4e-6d59-4729-b152-4f65bde5e975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:33.285951 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400b9b4e-6d59-4729-b152-4f65bde5e975 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:33.286369 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400b9b4e-6d59-4729-b152-4f65bde5e975 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:33.286777 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400b9b4e-6d59-4729-b152-4f65bde5e975 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:33.287614 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400b9b4e-6d59-4729-b152-4f65bde5e975 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:33.289256 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400b9b4e-6d59-4729-b152-4f65bde5e975 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:33.303015 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 454/902] 10.222.0.40 () {56 vars in 1087 bytes} [Fri Aug 5 10:14:33 2022] GET /identity/v3/services => generated 1268 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:14:33.958423 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f21e7-f46e-4b07-abc9-6e34b25e7a45 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:33.959015 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f21e7-f46e-4b07-abc9-6e34b25e7a45 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:33.959015 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f21e7-f46e-4b07-abc9-6e34b25e7a45 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:33.960628 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 449/903] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:33.966389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690bf47c-ee2b-415a-ad08-8777602ef663 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:33.966818 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690bf47c-ee2b-415a-ad08-8777602ef663 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:33.967011 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690bf47c-ee2b-415a-ad08-8777602ef663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:33.986740 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-690bf47c-ee2b-415a-ad08-8777602ef663 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:33.987989 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-690bf47c-ee2b-415a-ad08-8777602ef663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:34.057137 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 455/904] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:33 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:14:34.080224 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475f21e7-f46e-4b07-abc9-6e34b25e7a45 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:34.166941 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:34.188483 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475f21e7-f46e-4b07-abc9-6e34b25e7a45 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:34.222883 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d399cb-5af1-4bae-8e3a-a340054389b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:34.223578 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d399cb-5af1-4bae-8e3a-a340054389b1 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:34.223864 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d399cb-5af1-4bae-8e3a-a340054389b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:34.224138 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d399cb-5af1-4bae-8e3a-a340054389b1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:34.224820 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d399cb-5af1-4bae-8e3a-a340054389b1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:34.226246 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d399cb-5af1-4bae-8e3a-a340054389b1 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:34.235662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-b3d399cb-5af1-4bae-8e3a-a340054389b1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 10:14:34.235662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:34.235662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 10:14:34.235662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 10:14:34.235662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 10:14:34.235662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 10:14:34.235662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 10:14:34.235662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:34.235662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 10:14:34.235662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:34.235662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 10:14:34.235662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:34.235662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:34.235662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:34.235662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:34.235662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:34.236585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:34.236585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:34.236585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:34.236585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:34.236585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:34.236585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:34.236585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:34.236585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 10:14:34.236585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 10:14:34.236585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 10:14:34.236585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 10:14:34.236585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:34.236585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:34.236585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:34.236585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:34.236585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:34.237420 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:34.237420 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:34.237420 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:34.237420 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:34.237420 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:34.237420 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:34.237420 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:34.237420 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:34.237420 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:34.237420 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:34.237420 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:34.237420 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 10:14:34.237420 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:34.237420 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 10:14:34.237420 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:34.237420 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 450/905] 10.222.0.40 () {56 vars in 1132 bytes} [Fri Aug 5 10:14:34 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:34.243045 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690bf47c-ee2b-415a-ad08-8777602ef663 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:34.325073 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:34.346807 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690bf47c-ee2b-415a-ad08-8777602ef663 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:34.380870 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb61e3-9e0e-42a9-9f93-b32c7058db95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:34.381491 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb61e3-9e0e-42a9-9f93-b32c7058db95 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:34.381764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb61e3-9e0e-42a9-9f93-b32c7058db95 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:34.382031 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb61e3-9e0e-42a9-9f93-b32c7058db95 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:34.382707 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eb61e3-9e0e-42a9-9f93-b32c7058db95 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:34.383041 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eb61e3-9e0e-42a9-9f93-b32c7058db95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:34.384416 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eb61e3-9e0e-42a9-9f93-b32c7058db95 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:34.389797 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 456/906] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Aug 5 10:14:34 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:34.396201 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d399cb-5af1-4bae-8e3a-a340054389b1 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:34.498816 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d399cb-5af1-4bae-8e3a-a340054389b1 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:34.531899 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7a513-54cf-4c13-bcf5-729b0caae835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:34.532519 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7a513-54cf-4c13-bcf5-729b0caae835 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:34.532793 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7a513-54cf-4c13-bcf5-729b0caae835 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:34.533058 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7a513-54cf-4c13-bcf5-729b0caae835 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:34.533732 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b7a513-54cf-4c13-bcf5-729b0caae835 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:34.534030 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b7a513-54cf-4c13-bcf5-729b0caae835 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:34.535430 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b7a513-54cf-4c13-bcf5-729b0caae835 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:34.540535 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 451/907] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Aug 5 10:14:34 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:34.548081 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb61e3-9e0e-42a9-9f93-b32c7058db95 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:34.650913 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb61e3-9e0e-42a9-9f93-b32c7058db95 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:34.690857 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351897c6-a53f-416b-9009-4ad30491d7f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:34.691525 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351897c6-a53f-416b-9009-4ad30491d7f2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:34.691802 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351897c6-a53f-416b-9009-4ad30491d7f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:34.692069 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351897c6-a53f-416b-9009-4ad30491d7f2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:34.692819 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351897c6-a53f-416b-9009-4ad30491d7f2 None admin] RBAC: Adding query filter params (interface=public, service_id=739ba599694b45bdb05bd79a9ce940db, region_id=RegionOne) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:34.693122 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351897c6-a53f-416b-9009-4ad30491d7f2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:34.694499 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351897c6-a53f-416b-9009-4ad30491d7f2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:34.700170 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 457/908] 10.222.0.40 () {56 vars in 1251 bytes} [Fri Aug 5 10:14:34 2022] GET /identity/v3/endpoints?service_id=739ba599694b45bdb05bd79a9ce940db&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:34.705523 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7a513-54cf-4c13-bcf5-729b0caae835 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:34.709431 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:34.809303 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7a513-54cf-4c13-bcf5-729b0caae835 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:34.847802 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f82c7-4d9b-448d-9777-b9e29117b1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:34.848435 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f82c7-4d9b-448d-9777-b9e29117b1f6 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:34.848713 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f82c7-4d9b-448d-9777-b9e29117b1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:34.848984 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f82c7-4d9b-448d-9777-b9e29117b1f6 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:34.849638 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179f82c7-4d9b-448d-9777-b9e29117b1f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:34.851040 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179f82c7-4d9b-448d-9777-b9e29117b1f6 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:34.857324 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 452/909] 10.222.0.40 () {56 vars in 1087 bytes} [Fri Aug 5 10:14:34 2022] GET /identity/v3/services => generated 1268 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:14:34.893968 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f0d6c-41da-460b-822a-153dfff9292a None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:34.894617 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f0d6c-41da-460b-822a-153dfff9292a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:34.895123 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f0d6c-41da-460b-822a-153dfff9292a None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:34.897114 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 458/910] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:34.903314 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87ef95-e393-4b08-a857-db11e3ef2df2 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:34.903954 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87ef95-e393-4b08-a857-db11e3ef2df2 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:34.904430 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87ef95-e393-4b08-a857-db11e3ef2df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:34.924990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-6a87ef95-e393-4b08-a857-db11e3ef2df2 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:34.926585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6a87ef95-e393-4b08-a857-db11e3ef2df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:34.999122 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:35.005593 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 453/911] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:34 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:14:35.028825 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0f0d6c-41da-460b-822a-153dfff9292a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:35.038349 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:35.139532 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0f0d6c-41da-460b-822a-153dfff9292a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:35.183311 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35251b80-db33-4698-a55c-01a652a38463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:35.183943 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35251b80-db33-4698-a55c-01a652a38463 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:35.184228 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35251b80-db33-4698-a55c-01a652a38463 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:35.184499 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35251b80-db33-4698-a55c-01a652a38463 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:35.185203 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35251b80-db33-4698-a55c-01a652a38463 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:35.186586 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35251b80-db33-4698-a55c-01a652a38463 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:35.204712 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 459/912] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 5 10:14:35 2022] POST /identity/v3/services => generated 241 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:14:35.257919 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc86da5-de78-448a-a290-dbd3f59a766b None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:35.258210 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc86da5-de78-448a-a290-dbd3f59a766b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:35.258460 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc86da5-de78-448a-a290-dbd3f59a766b None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:35.260238 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 454/913] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:35.266316 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962397f-f250-40b9-a688-10861a3dcf10 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:35.266631 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962397f-f250-40b9-a688-10861a3dcf10 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:35.266878 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962397f-f250-40b9-a688-10861a3dcf10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:35.288450 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9962397f-f250-40b9-a688-10861a3dcf10 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:35.289745 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9962397f-f250-40b9-a688-10861a3dcf10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:35.366751 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:35.375790 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 460/914] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:14:35.390169 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768efdc0-107f-4323-8077-d8fe14cb7500 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:35.390439 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768efdc0-107f-4323-8077-d8fe14cb7500 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:35.390648 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768efdc0-107f-4323-8077-d8fe14cb7500 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:35.393318 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 455/915] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:35.397663 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676980aa-6199-44b0-aba6-937736c68dfb None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:35.397945 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676980aa-6199-44b0-aba6-937736c68dfb None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:35.398220 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676980aa-6199-44b0-aba6-937736c68dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:35.400124 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768efdc0-107f-4323-8077-d8fe14cb7500 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:35.417729 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-676980aa-6199-44b0-aba6-937736c68dfb None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:35.419546 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-676980aa-6199-44b0-aba6-937736c68dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:35.492119 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 461/916] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:14:35.518208 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768efdc0-107f-4323-8077-d8fe14cb7500 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:35.519686 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676980aa-6199-44b0-aba6-937736c68dfb None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:35.559175 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd151681-683f-4363-a815-74b42ae74f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:35.559891 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd151681-683f-4363-a815-74b42ae74f8c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:35.560175 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd151681-683f-4363-a815-74b42ae74f8c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:35.560452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd151681-683f-4363-a815-74b42ae74f8c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:35.575552 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd151681-683f-4363-a815-74b42ae74f8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:14:35.582433 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-cd151681-683f-4363-a815-74b42ae74f8c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 10:14:35.582433 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:35.582433 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:35.582433 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:35.582433 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:35.582433 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:35.582433 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:35.582433 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:35.582433 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:35.582433 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:35.582433 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:35.582433 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:35.582433 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:14:35.582433 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:14:35.582433 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:14:35.582433 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:14:35.583404 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:14:35.583404 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:14:35.583404 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:14:35.583404 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:14:35.583404 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:35.583404 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:35.583404 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:35.583404 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:35.583404 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:35.583404 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:35.583404 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:35.583404 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:35.583404 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:35.583404 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:35.583404 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:35.583404 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:35.584257 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:35.584257 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:35.584257 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:35.584257 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:35.584257 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:14:35.584257 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:14:35.584257 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:14:35.584257 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:14:35.584257 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:14:35.584257 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:14:35.584257 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 10:14:35.584257 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:35.584257 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 456/917] 10.222.0.40 () {56 vars in 1120 bytes} [Fri Aug 5 10:14:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:35.590069 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd151681-683f-4363-a815-74b42ae74f8c None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:35.632607 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676980aa-6199-44b0-aba6-937736c68dfb None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:35.659644 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:35.673683 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18baef3-a51d-4f8b-a86a-acf5a6f9d338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:35.674752 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18baef3-a51d-4f8b-a86a-acf5a6f9d338 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:35.675252 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18baef3-a51d-4f8b-a86a-acf5a6f9d338 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:35.675761 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18baef3-a51d-4f8b-a86a-acf5a6f9d338 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:35.676631 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18baef3-a51d-4f8b-a86a-acf5a6f9d338 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:35.678221 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18baef3-a51d-4f8b-a86a-acf5a6f9d338 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:35.679335 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:35.688581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-f18baef3-a51d-4f8b-a86a-acf5a6f9d338 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 10:14:35.688581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:35.688581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 10:14:35.688581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 10:14:35.688581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 10:14:35.688581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 10:14:35.688581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 10:14:35.688581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:35.688581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 10:14:35.688581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:35.688581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 10:14:35.688581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:35.688581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:35.688581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:35.688581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:35.688581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:35.688581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:35.689628 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:35.689628 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:35.689628 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:35.689628 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:35.689628 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:35.689628 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:35.689628 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 10:14:35.689628 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 10:14:35.689628 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 10:14:35.689628 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 10:14:35.689628 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:35.689628 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:35.689628 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:35.689628 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:35.689628 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:35.689628 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:35.690467 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:35.690467 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:35.690467 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:35.690467 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:35.690467 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:35.690467 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:35.690467 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:35.690467 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:35.690467 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:35.690467 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:35.690467 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 10:14:35.690467 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:35.690467 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 10:14:35.690467 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:35.693564 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 462/918] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 5 10:14:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:14:35.700404 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18baef3-a51d-4f8b-a86a-acf5a6f9d338 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:35.714501 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd151681-683f-4363-a815-74b42ae74f8c None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:35.751737 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66bfc5a-1719-44e2-8775-420ac9616b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:35.752748 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66bfc5a-1719-44e2-8775-420ac9616b93 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:35.753229 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66bfc5a-1719-44e2-8775-420ac9616b93 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:35.753693 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66bfc5a-1719-44e2-8775-420ac9616b93 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:35.754836 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66bfc5a-1719-44e2-8775-420ac9616b93 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:35.755242 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66bfc5a-1719-44e2-8775-420ac9616b93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:35.756691 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66bfc5a-1719-44e2-8775-420ac9616b93 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:35.766255 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 457/919] 10.222.0.40 () {56 vars in 1115 bytes} [Fri Aug 5 10:14:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:35.772807 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66bfc5a-1719-44e2-8775-420ac9616b93 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:35.823951 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18baef3-a51d-4f8b-a86a-acf5a6f9d338 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:35.873009 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474960ba-aae1-4e4f-8d67-70194c90dcef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:35.874179 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474960ba-aae1-4e4f-8d67-70194c90dcef None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:35.874666 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474960ba-aae1-4e4f-8d67-70194c90dcef None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:35.875377 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474960ba-aae1-4e4f-8d67-70194c90dcef None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:35.876545 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474960ba-aae1-4e4f-8d67-70194c90dcef None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:35.877035 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474960ba-aae1-4e4f-8d67-70194c90dcef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:35.878632 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474960ba-aae1-4e4f-8d67-70194c90dcef None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:35.886626 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 463/920] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 5 10:14:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:14:35.892313 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474960ba-aae1-4e4f-8d67-70194c90dcef None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:35.895889 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66bfc5a-1719-44e2-8775-420ac9616b93 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:35.921418 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:35.941040 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee343c5-3f45-49d6-96fb-7b53c443cbad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:35.942349 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee343c5-3f45-49d6-96fb-7b53c443cbad None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:35.942728 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee343c5-3f45-49d6-96fb-7b53c443cbad None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:35.943128 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee343c5-3f45-49d6-96fb-7b53c443cbad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:35.953714 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:35.960232 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee343c5-3f45-49d6-96fb-7b53c443cbad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:35.962591 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee343c5-3f45-49d6-96fb-7b53c443cbad None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:35.982301 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-eee343c5-3f45-49d6-96fb-7b53c443cbad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:35.982301 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:35.982301 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:35.982301 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:35.982301 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:35.982301 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:35.982301 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:35.982301 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:35.982301 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:35.982301 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:35.982301 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:35.982301 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:35.982301 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:35.982301 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:35.982301 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:35.983450 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:35.983450 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:35.983450 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:35.983450 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:35.983450 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:35.983450 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:35.983450 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:35.983450 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:35.983450 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:35.983450 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:35.983450 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:35.983450 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:35.983450 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:35.983450 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:35.983450 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:35.983450 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:35.983450 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:35.984542 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:35.984542 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:35.984542 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:35.984542 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:35.986563 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 458/921] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:35.992365 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee343c5-3f45-49d6-96fb-7b53c443cbad None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:36.023295 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474960ba-aae1-4e4f-8d67-70194c90dcef None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:36.060923 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59a33a-067b-4d95-97f1-66ce0002af58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:36.061558 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59a33a-067b-4d95-97f1-66ce0002af58 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:36.061864 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59a33a-067b-4d95-97f1-66ce0002af58 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:36.062207 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59a33a-067b-4d95-97f1-66ce0002af58 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:36.062988 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c59a33a-067b-4d95-97f1-66ce0002af58 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:36.063405 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c59a33a-067b-4d95-97f1-66ce0002af58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:36.065083 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c59a33a-067b-4d95-97f1-66ce0002af58 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:36.076345 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 464/922] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 5 10:14:35 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:36.078474 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59a33a-067b-4d95-97f1-66ce0002af58 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:36.113614 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee343c5-3f45-49d6-96fb-7b53c443cbad None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:36.150611 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f9d020-efd9-4c58-ad1a-34b79917fcd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:36.152182 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9d020-efd9-4c58-ad1a-34b79917fcd9 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:36.152454 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9d020-efd9-4c58-ad1a-34b79917fcd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:36.152717 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9d020-efd9-4c58-ad1a-34b79917fcd9 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:36.153409 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f9d020-efd9-4c58-ad1a-34b79917fcd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:36.153708 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f9d020-efd9-4c58-ad1a-34b79917fcd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:36.155110 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f9d020-efd9-4c58-ad1a-34b79917fcd9 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:36.174768 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 459/923] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:14:36.176630 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f9d020-efd9-4c58-ad1a-34b79917fcd9 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:36.210352 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:36.230367 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59a33a-067b-4d95-97f1-66ce0002af58 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:36.247794 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:36.270236 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee580b83-ac36-4fd7-8946-3c2e292bf6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:36.271030 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee580b83-ac36-4fd7-8946-3c2e292bf6ae None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:36.271701 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee580b83-ac36-4fd7-8946-3c2e292bf6ae None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:36.271856 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee580b83-ac36-4fd7-8946-3c2e292bf6ae None admin] PATH_INFO: `/v3/endpoints` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:36.272779 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee580b83-ac36-4fd7-8946-3c2e292bf6ae None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:36.274384 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee580b83-ac36-4fd7-8946-3c2e292bf6ae None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:36.294949 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f9d020-efd9-4c58-ad1a-34b79917fcd9 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:36.302949 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 465/924] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:14:36 2022] POST /identity/v3/endpoints => generated 346 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:14:36.352429 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a05d9-7475-48c8-af41-aaee19d1458d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:36.353556 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a05d9-7475-48c8-af41-aaee19d1458d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:36.354027 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a05d9-7475-48c8-af41-aaee19d1458d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:36.355303 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a05d9-7475-48c8-af41-aaee19d1458d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:36.364662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872a05d9-7475-48c8-af41-aaee19d1458d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:36.366550 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872a05d9-7475-48c8-af41-aaee19d1458d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:36.380751 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-872a05d9-7475-48c8-af41-aaee19d1458d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 10:14:36.380751 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:36.380751 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:36.380751 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:36.380751 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:36.380751 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:36.380751 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:36.380751 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:36.380751 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:36.380751 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:36.380751 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:36.380751 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:36.380751 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 10:14:36.380751 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 10:14:36.380751 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 10:14:36.380751 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 10:14:36.381768 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:36.381768 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:36.381768 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 10:14:36.381768 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:36.381768 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 10:14:36.381768 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:36.381768 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:36.381768 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:36.381768 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:36.381768 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:36.381768 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:36.381768 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:36.381768 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:36.381768 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:36.381768 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:36.381768 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:36.381768 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:36.381768 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:36.382734 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:36.382734 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:36.382734 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 10:14:36.382734 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 10:14:36.382734 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 10:14:36.382734 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 10:14:36.382734 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 10:14:36.382734 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 10:14:36.382734 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 10:14:36.382734 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:36.385967 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 460/925] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Aug 5 10:14:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:14:36.391707 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee580b83-ac36-4fd7-8946-3c2e292bf6ae None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:36.522666 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee580b83-ac36-4fd7-8946-3c2e292bf6ae None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:36.557870 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfeebcd-9392-44ea-8bf5-bae6591251ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:36.558750 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfeebcd-9392-44ea-8bf5-bae6591251ae None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:36.559332 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfeebcd-9392-44ea-8bf5-bae6591251ae None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:36.559766 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfeebcd-9392-44ea-8bf5-bae6591251ae None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:36.560723 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfeebcd-9392-44ea-8bf5-bae6591251ae None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:36.561169 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfeebcd-9392-44ea-8bf5-bae6591251ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:36.562756 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfeebcd-9392-44ea-8bf5-bae6591251ae None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:36.581215 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 466/926] 10.222.0.40 () {56 vars in 1149 bytes} [Fri Aug 5 10:14:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:14:36.586701 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a05d9-7475-48c8-af41-aaee19d1458d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:36.684448 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:36.704307 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a05d9-7475-48c8-af41-aaee19d1458d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:36.738999 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c7f68-36b0-4ad4-bce8-be8304114f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:36.739929 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c7f68-36b0-4ad4-bce8-be8304114f75 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:36.740363 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c7f68-36b0-4ad4-bce8-be8304114f75 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:36.740771 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c7f68-36b0-4ad4-bce8-be8304114f75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:36.752464 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69c7f68-36b0-4ad4-bce8-be8304114f75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:36.754317 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69c7f68-36b0-4ad4-bce8-be8304114f75 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:36.769582 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-e69c7f68-36b0-4ad4-bce8-be8304114f75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:36.769582 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:36.769582 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:36.769582 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:36.769582 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:36.769582 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:36.769582 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:36.769582 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:36.769582 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:36.769582 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:36.769582 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:36.769582 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:36.769582 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:36.769582 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:36.769582 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:36.770711 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:36.770711 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:36.770711 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:36.770711 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:36.770711 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:36.770711 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:36.770711 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:36.770711 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:36.770711 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:36.770711 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:36.770711 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:36.770711 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:36.770711 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:36.770711 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:36.770711 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:36.770711 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:36.770711 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:36.771856 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:36.771856 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:36.771856 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:36.771856 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:36.773852 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 461/927] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:36.780021 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfeebcd-9392-44ea-8bf5-bae6591251ae None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:36.810665 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:36.892480 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfeebcd-9392-44ea-8bf5-bae6591251ae None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:36.928309 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f39d5a-9804-4174-8c50-8d1fb9cad00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:36.928945 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f39d5a-9804-4174-8c50-8d1fb9cad00b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:36.929224 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f39d5a-9804-4174-8c50-8d1fb9cad00b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:36.929500 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f39d5a-9804-4174-8c50-8d1fb9cad00b None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:36.930267 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f39d5a-9804-4174-8c50-8d1fb9cad00b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:36.930575 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f39d5a-9804-4174-8c50-8d1fb9cad00b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:36.931997 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f39d5a-9804-4174-8c50-8d1fb9cad00b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:36.944385 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 467/928] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:36.950156 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c7f68-36b0-4ad4-bce8-be8304114f75 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:36.998840 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735b3db-7c31-476a-aa32-e1aa434241a0 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:36.999216 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735b3db-7c31-476a-aa32-e1aa434241a0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:36.999446 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735b3db-7c31-476a-aa32-e1aa434241a0 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:37.001074 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 468/929] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 10:14:37.006138 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b7e15-f8c4-40cc-8772-fbf30687c267 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:37.006426 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b7e15-f8c4-40cc-8772-fbf30687c267 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:37.006706 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b7e15-f8c4-40cc-8772-fbf30687c267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:37.026291 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e97b7e15-f8c4-40cc-8772-fbf30687c267 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:37.027532 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e97b7e15-f8c4-40cc-8772-fbf30687c267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:37.071747 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c7f68-36b0-4ad4-bce8-be8304114f75 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:37.103212 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 469/930] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 10:14:37.108869 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f2d71-8e63-4a7f-a281-e357e19681dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:37.109764 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f2d71-8e63-4a7f-a281-e357e19681dc None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:37.110261 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f2d71-8e63-4a7f-a281-e357e19681dc None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:37.110730 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f2d71-8e63-4a7f-a281-e357e19681dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:37.118071 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9f2d71-8e63-4a7f-a281-e357e19681dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:37.121708 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97b7e15-f8c4-40cc-8772-fbf30687c267 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:37.123277 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9f2d71-8e63-4a7f-a281-e357e19681dc None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:37.126483 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:37.137178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-fc9f2d71-8e63-4a7f-a281-e357e19681dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:37.137178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:37.137178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:37.137178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:37.137178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:37.137178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:37.137178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:37.137178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:37.137178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:37.137178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:37.137178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:37.137178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:37.137178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 10:14:37.137178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 10:14:37.137178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 10:14:37.138104 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 10:14:37.138104 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:37.138104 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:37.138104 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:37.138104 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:37.138104 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:37.138104 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:37.138104 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:37.138104 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:37.138104 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:37.138104 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:37.138104 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:37.138104 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:37.138104 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:37.138104 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:37.138104 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:37.138104 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:37.139551 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 10:14:37.139551 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 10:14:37.139551 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 10:14:37.139551 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 10:14:37.139551 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 10:14:37.139551 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 10:14:37.139551 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:37.139551 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:37.139551 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 462/931] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:14:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:37.148298 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:37.152213 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f2d71-8e63-4a7f-a281-e357e19681dc None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:37.250238 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97b7e15-f8c4-40cc-8772-fbf30687c267 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:37.260791 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f2d71-8e63-4a7f-a281-e357e19681dc None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:37.288624 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f22a73f-549f-4563-96c0-61a332f56f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:37.289601 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22a73f-549f-4563-96c0-61a332f56f17 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:37.290152 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22a73f-549f-4563-96c0-61a332f56f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:37.290606 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22a73f-549f-4563-96c0-61a332f56f17 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:37.291453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f22a73f-549f-4563-96c0-61a332f56f17 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:37.293652 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f22a73f-549f-4563-96c0-61a332f56f17 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:37.295664 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a553ea-44b8-45f5-b05c-f96fe991cd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:37.296380 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a553ea-44b8-45f5-b05c-f96fe991cd47 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:37.296726 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a553ea-44b8-45f5-b05c-f96fe991cd47 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:37.297063 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a553ea-44b8-45f5-b05c-f96fe991cd47 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:37.297881 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a553ea-44b8-45f5-b05c-f96fe991cd47 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:37.298252 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a553ea-44b8-45f5-b05c-f96fe991cd47 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:37.299800 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a553ea-44b8-45f5-b05c-f96fe991cd47 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:37.308598 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-4f22a73f-549f-4563-96c0-61a332f56f17 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 10:14:37.308598 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:37.308598 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 10:14:37.308598 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 10:14:37.308598 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 10:14:37.308598 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 10:14:37.308598 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 10:14:37.308598 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:37.308598 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 10:14:37.308598 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:37.308598 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 10:14:37.308598 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:37.308598 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:37.308598 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:37.308598 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:37.308598 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:37.309632 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:37.309632 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:37.309632 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:37.309632 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:37.309632 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:37.309632 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:37.309632 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:37.309632 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 10:14:37.309632 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 10:14:37.309632 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 10:14:37.309632 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 10:14:37.309632 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:37.309632 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:37.309632 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:37.309632 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:37.309632 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:37.310494 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:37.310494 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:37.310494 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:37.310494 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:37.310494 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:37.310494 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:37.310494 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:37.310494 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:37.310494 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:37.310494 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:37.310494 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:37.310494 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 10:14:37.310494 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:37.310494 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 10:14:37.310494 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:37.313316 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 470/932] 10.222.0.40 () {56 vars in 1132 bytes} [Fri Aug 5 10:14:37 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:14:37.314985 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 463/933] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 10:14:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:37.319386 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a553ea-44b8-45f5-b05c-f96fe991cd47 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:37.319511 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f22a73f-549f-4563-96c0-61a332f56f17 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:37.380113 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:37.409508 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:37.422633 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a553ea-44b8-45f5-b05c-f96fe991cd47 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:37.429723 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f22a73f-549f-4563-96c0-61a332f56f17 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:37.457641 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4025b64-2b7a-45a6-b0f2-b9900ca7a7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:37.458315 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4025b64-2b7a-45a6-b0f2-b9900ca7a7e0 None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:37.458597 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4025b64-2b7a-45a6-b0f2-b9900ca7a7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:37.458875 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4025b64-2b7a-45a6-b0f2-b9900ca7a7e0 None admin] PATH_INFO: `/v3/projects/36b8b505012d4d0bac54845b6799e743/users/52fc3ba90dc045c2a5b7bb730da96acc/roles/f10badb1efd340eda8d6358dc6bd9c28` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:37.461139 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4025b64-2b7a-45a6-b0f2-b9900ca7a7e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=36b8b505012d4d0bac54845b6799e743, user_id=52fc3ba90dc045c2a5b7bb730da96acc, role_id=f10badb1efd340eda8d6358dc6bd9c28)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:37.462582 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4025b64-2b7a-45a6-b0f2-b9900ca7a7e0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:37.467106 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e5f25-5cb4-4bc5-aead-b0da3b35dd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:37.467736 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e5f25-5cb4-4bc5-aead-b0da3b35dd8c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:37.467995 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e5f25-5cb4-4bc5-aead-b0da3b35dd8c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:37.468283 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e5f25-5cb4-4bc5-aead-b0da3b35dd8c None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:37.468941 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5e5f25-5cb4-4bc5-aead-b0da3b35dd8c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:37.469233 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5e5f25-5cb4-4bc5-aead-b0da3b35dd8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:37.470628 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5e5f25-5cb4-4bc5-aead-b0da3b35dd8c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:37.483172 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 471/934] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Aug 5 10:14:37 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:37.483172 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174979c-5d91-4c2b-aafb-1704d39dee87 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:37.483172 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174979c-5d91-4c2b-aafb-1704d39dee87 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:37.483172 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174979c-5d91-4c2b-aafb-1704d39dee87 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:37.483172 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 472/936] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:37.483172 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6174979c-5d91-4c2b-aafb-1704d39dee87 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:37.487074 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 464/936] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 10:14:37 2022] PUT /identity/v3/projects/36b8b505012d4d0bac54845b6799e743/users/52fc3ba90dc045c2a5b7bb730da96acc/roles/f10badb1efd340eda8d6358dc6bd9c28 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:14:37.488637 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7074ed9b-796a-4c74-a62c-76fee8fadc34 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:37.488933 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7074ed9b-796a-4c74-a62c-76fee8fadc34 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:37.489212 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7074ed9b-796a-4c74-a62c-76fee8fadc34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:37.524808 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-7074ed9b-796a-4c74-a62c-76fee8fadc34 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:37.527542 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7074ed9b-796a-4c74-a62c-76fee8fadc34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:37.607301 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6174979c-5d91-4c2b-aafb-1704d39dee87 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:37.639161 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 465/937] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:14:37.656085 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237368f3-8410-46f6-8fce-804ade8ec284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:37.656731 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237368f3-8410-46f6-8fce-804ade8ec284 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:37.657008 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237368f3-8410-46f6-8fce-804ade8ec284 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:37.657279 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237368f3-8410-46f6-8fce-804ade8ec284 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:37.657975 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237368f3-8410-46f6-8fce-804ade8ec284 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:37.658280 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237368f3-8410-46f6-8fce-804ade8ec284 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:37.658666 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7074ed9b-796a-4c74-a62c-76fee8fadc34 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:37.659747 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237368f3-8410-46f6-8fce-804ade8ec284 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:37.670032 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 473/938] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Aug 5 10:14:37 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:37.676917 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237368f3-8410-46f6-8fce-804ade8ec284 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:37.690791 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:37.730705 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:37.795547 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237368f3-8410-46f6-8fce-804ade8ec284 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:37.799802 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7074ed9b-796a-4c74-a62c-76fee8fadc34 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:37.835915 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d8172-56b1-4007-9156-5cde464d4ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:37.836955 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d8172-56b1-4007-9156-5cde464d4ffc None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:37.837469 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d8172-56b1-4007-9156-5cde464d4ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:37.837966 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d8172-56b1-4007-9156-5cde464d4ffc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:37.838706 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541850a2-f6be-492c-b96d-186f06daba12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:37.839118 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394d8172-56b1-4007-9156-5cde464d4ffc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:37.839416 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541850a2-f6be-492c-b96d-186f06daba12 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:37.839691 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541850a2-f6be-492c-b96d-186f06daba12 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:37.839960 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541850a2-f6be-492c-b96d-186f06daba12 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:37.840670 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541850a2-f6be-492c-b96d-186f06daba12 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:37.841599 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394d8172-56b1-4007-9156-5cde464d4ffc None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:37.842023 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541850a2-f6be-492c-b96d-186f06daba12 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:37.851587 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-394d8172-56b1-4007-9156-5cde464d4ffc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 10:14:37.851587 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:37.851587 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 10:14:37.851587 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 10:14:37.851587 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 10:14:37.851587 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 10:14:37.851587 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 10:14:37.851587 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:37.851587 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 10:14:37.851587 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:37.851587 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 10:14:37.851587 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:37.851587 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:37.851587 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:37.851587 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:37.851587 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:37.851587 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:37.852624 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:37.852624 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:37.852624 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:37.852624 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:37.852624 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:37.852624 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:37.852624 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 10:14:37.852624 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 10:14:37.852624 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 10:14:37.852624 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 10:14:37.852624 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:37.852624 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:37.852624 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:37.852624 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:37.852624 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:37.852624 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:37.853532 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:37.853532 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:37.853532 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:37.853532 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:37.853532 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:37.853532 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:37.853532 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:37.853532 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:37.853532 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:37.853532 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:37.853532 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 10:14:37.853532 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:37.853532 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 10:14:37.853532 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:37.859326 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 466/939] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Aug 5 10:14:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:14:37.861784 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d8172-56b1-4007-9156-5cde464d4ffc None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:37.875853 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 474/940] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:14:37 2022] POST /identity/v3/endpoints => generated 347 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:14:37.988037 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d8172-56b1-4007-9156-5cde464d4ffc None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:38.008177 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:38.037378 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d920ef-762e-430e-bfc8-6aadf31c66dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:38.038007 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d920ef-762e-430e-bfc8-6aadf31c66dd None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:38.038282 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d920ef-762e-430e-bfc8-6aadf31c66dd None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:38.038550 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d920ef-762e-430e-bfc8-6aadf31c66dd None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:38.039267 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d920ef-762e-430e-bfc8-6aadf31c66dd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:38.039565 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d920ef-762e-430e-bfc8-6aadf31c66dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:38.040955 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d920ef-762e-430e-bfc8-6aadf31c66dd None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:38.047238 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 467/941] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Aug 5 10:14:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:14:38.053190 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541850a2-f6be-492c-b96d-186f06daba12 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:38.120716 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:38.170012 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541850a2-f6be-492c-b96d-186f06daba12 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:38.204965 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94012e-9a20-45c3-bb54-61b08d925153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:38.205961 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94012e-9a20-45c3-bb54-61b08d925153 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:38.206431 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94012e-9a20-45c3-bb54-61b08d925153 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:38.206971 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94012e-9a20-45c3-bb54-61b08d925153 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:38.207867 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f94012e-9a20-45c3-bb54-61b08d925153 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:38.208379 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f94012e-9a20-45c3-bb54-61b08d925153 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:38.210009 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f94012e-9a20-45c3-bb54-61b08d925153 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:38.216371 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 475/942] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Aug 5 10:14:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:38.222651 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d920ef-762e-430e-bfc8-6aadf31c66dd None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:38.325387 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d920ef-762e-430e-bfc8-6aadf31c66dd None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:38.359587 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7cf010-c922-4837-af07-e98d07573886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:38.360554 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7cf010-c922-4837-af07-e98d07573886 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:38.361029 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7cf010-c922-4837-af07-e98d07573886 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:38.361521 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7cf010-c922-4837-af07-e98d07573886 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:38.362507 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7cf010-c922-4837-af07-e98d07573886 None admin] RBAC: Adding query filter params (interface=public, service_id=4c5766718189433a82fa0f7d7a84143f, region_id=RegionOne) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:38.363045 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7cf010-c922-4837-af07-e98d07573886 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:38.364597 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7cf010-c922-4837-af07-e98d07573886 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:38.372745 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 468/943] 10.222.0.40 () {56 vars in 1251 bytes} [Fri Aug 5 10:14:38 2022] GET /identity/v3/endpoints?service_id=4c5766718189433a82fa0f7d7a84143f&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:38.378675 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94012e-9a20-45c3-bb54-61b08d925153 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:38.459492 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d59b47-bc0c-4fb9-bbd2-4ad3d4ab0341 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:38.460147 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d59b47-bc0c-4fb9-bbd2-4ad3d4ab0341 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:38.460646 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d59b47-bc0c-4fb9-bbd2-4ad3d4ab0341 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:38.462588 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 469/944] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 10:14:38.467901 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e22758-5cb8-43e6-ba59-090ea7a701ed None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:38.468550 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e22758-5cb8-43e6-ba59-090ea7a701ed None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:38.469012 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e22758-5cb8-43e6-ba59-090ea7a701ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:38.488447 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94012e-9a20-45c3-bb54-61b08d925153 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:38.488989 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-53e22758-5cb8-43e6-ba59-090ea7a701ed None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:38.490692 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-53e22758-5cb8-43e6-ba59-090ea7a701ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:38.523779 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:38.527330 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:38.535811 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a8561f-2d84-4cb2-b4ce-3470e75719b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:38.537243 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a8561f-2d84-4cb2-b4ce-3470e75719b1 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:38.537709 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a8561f-2d84-4cb2-b4ce-3470e75719b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:38.538156 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a8561f-2d84-4cb2-b4ce-3470e75719b1 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:38.539496 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a8561f-2d84-4cb2-b4ce-3470e75719b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:38.541799 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a8561f-2d84-4cb2-b4ce-3470e75719b1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:38.550781 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 476/945] 10.222.0.40 () {56 vars in 1087 bytes} [Fri Aug 5 10:14:38 2022] GET /identity/v3/services => generated 1497 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:14:38.584992 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 470/946] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:14:38.609092 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a8561f-2d84-4cb2-b4ce-3470e75719b1 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:38.731659 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a8561f-2d84-4cb2-b4ce-3470e75719b1 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:38.767844 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5063a479-e912-43fb-b1e2-52b568d8cd1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:38.768526 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063a479-e912-43fb-b1e2-52b568d8cd1c None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:38.768807 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063a479-e912-43fb-b1e2-52b568d8cd1c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:38.769135 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063a479-e912-43fb-b1e2-52b568d8cd1c None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:38.769827 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5063a479-e912-43fb-b1e2-52b568d8cd1c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:38.771306 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5063a479-e912-43fb-b1e2-52b568d8cd1c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:38.773945 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: WARNING keystone.server.flask.common [None req-5063a479-e912-43fb-b1e2-52b568d8cd1c None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 05 10:14:38.794303 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.sql.core [None req-5063a479-e912-43fb-b1e2-52b568d8cd1c None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 05 10:14:38.794303 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 05 10:14:38.794303 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [parameters: {'id': '9129bd7462b54022a8d7fbd761d9a67a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 05 10:14:38.794303 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80142) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 05 10:14:38.802213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-5063a479-e912-43fb-b1e2-52b568d8cd1c None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9129bd7462b54022a8d7fbd761d9a67a.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9129bd7462b54022a8d7fbd761d9a67a. Aug 05 10:14:38.802213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:38.802213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 05 10:14:38.802213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 05 10:14:38.802213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 05 10:14:38.802213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 05 10:14:38.802213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 05 10:14:38.802213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application result = self._query(query) Aug 05 10:14:38.802213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 05 10:14:38.802213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application conn.query(q) Aug 05 10:14:38.802213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 05 10:14:38.802213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 05 10:14:38.802213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 05 10:14:38.803043 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application result.read() Aug 05 10:14:38.803043 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 05 10:14:38.803043 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 05 10:14:38.803043 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 05 10:14:38.803043 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 05 10:14:38.803043 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 05 10:14:38.803043 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 05 10:14:38.803043 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 05 10:14:38.803043 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 05 10:14:38.803043 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 05 10:14:38.803043 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:38.803043 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 05 10:14:38.803043 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:38.803043 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:38.803043 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 05 10:14:38.803043 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 05 10:14:38.803043 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 05 10:14:38.803043 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 05 10:14:38.804010 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 05 10:14:38.804010 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application next(self.gen) Aug 05 10:14:38.804010 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 05 10:14:38.804010 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application yield resource Aug 05 10:14:38.804010 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 05 10:14:38.804010 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application next(self.gen) Aug 05 10:14:38.804010 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 05 10:14:38.804010 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self.session.rollback() Aug 05 10:14:38.804010 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 05 10:14:38.804010 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self.force_reraise() Aug 05 10:14:38.804010 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 05 10:14:38.804010 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise self.value Aug 05 10:14:38.804010 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 05 10:14:38.804010 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 05 10:14:38.804010 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 05 10:14:38.804010 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application session.commit() Aug 05 10:14:38.804010 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 05 10:14:38.804913 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 05 10:14:38.804913 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 05 10:14:38.804913 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._prepare_impl() Aug 05 10:14:38.804913 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 05 10:14:38.804913 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self.session.flush() Aug 05 10:14:38.804913 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 05 10:14:38.804913 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._flush(objects) Aug 05 10:14:38.804913 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 05 10:14:38.804913 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 05 10:14:38.804913 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 05 10:14:38.804913 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application compat.raise_( Aug 05 10:14:38.804913 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 05 10:14:38.804913 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception Aug 05 10:14:38.804913 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 05 10:14:38.804913 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application flush_context.execute() Aug 05 10:14:38.804913 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 05 10:14:38.804913 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rec.execute(self) Aug 05 10:14:38.805834 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 05 10:14:38.805834 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 05 10:14:38.805834 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 05 10:14:38.805834 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 05 10:14:38.805834 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 05 10:14:38.805834 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 05 10:14:38.805834 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 05 10:14:38.805834 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 05 10:14:38.805834 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 05 10:14:38.805834 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 05 10:14:38.805834 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 05 10:14:38.805834 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 05 10:14:38.805834 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 05 10:14:38.805834 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 05 10:14:38.805834 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 05 10:14:38.808285 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 05 10:14:38.808285 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 05 10:14:38.808285 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception Aug 05 10:14:38.808285 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 05 10:14:38.808285 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 05 10:14:38.808285 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 05 10:14:38.808285 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 05 10:14:38.808285 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 05 10:14:38.808285 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application result = self._query(query) Aug 05 10:14:38.808285 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 05 10:14:38.808285 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application conn.query(q) Aug 05 10:14:38.808285 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 05 10:14:38.808285 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 05 10:14:38.808285 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 05 10:14:38.808285 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application result.read() Aug 05 10:14:38.808285 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 05 10:14:38.808285 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 05 10:14:38.810371 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 05 10:14:38.810371 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 05 10:14:38.810371 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 05 10:14:38.810371 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 05 10:14:38.810371 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 05 10:14:38.810371 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 05 10:14:38.810371 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 05 10:14:38.810371 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 05 10:14:38.810371 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [parameters: {'id': '9129bd7462b54022a8d7fbd761d9a67a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 05 10:14:38.810371 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 05 10:14:38.810371 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:38.810371 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 10:14:38.810371 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:38.810371 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:38.810371 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 05 10:14:38.812245 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 05 10:14:38.812245 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 05 10:14:38.812245 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 05 10:14:38.812245 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 05 10:14:38.812245 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:38.812245 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 10:14:38.812245 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:38.812245 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:38.812245 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:38.812245 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:38.812245 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:38.812245 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:38.812245 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:38.812245 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:38.812245 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:38.812245 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:38.812245 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:38.814396 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:38.814396 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 05 10:14:38.814396 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 05 10:14:38.814396 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:38.814396 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:38.814396 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 05 10:14:38.814396 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 05 10:14:38.814396 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9129bd7462b54022a8d7fbd761d9a67a. Aug 05 10:14:38.814396 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:38.817431 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 477/947] 10.222.0.40 () {60 vars in 1140 bytes} [Fri Aug 5 10:14:38 2022] POST /identity/v3/projects => generated 262 bytes in 209 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:38.826955 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e22758-5cb8-43e6-ba59-090ea7a701ed None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:38.932837 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e22758-5cb8-43e6-ba59-090ea7a701ed None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:38.970264 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3900c699-8cf3-48a7-adce-7fc59a8b8eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:38.971198 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900c699-8cf3-48a7-adce-7fc59a8b8eaa None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:38.971625 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900c699-8cf3-48a7-adce-7fc59a8b8eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:38.972081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900c699-8cf3-48a7-adce-7fc59a8b8eaa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:38.979443 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3900c699-8cf3-48a7-adce-7fc59a8b8eaa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:38.981099 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3900c699-8cf3-48a7-adce-7fc59a8b8eaa None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:38.993289 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-3900c699-8cf3-48a7-adce-7fc59a8b8eaa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:38.993289 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:38.993289 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:38.993289 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:38.993289 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:38.993289 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:38.993289 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:38.993289 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:38.993289 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:38.993289 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:38.993289 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:38.993289 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:38.993289 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 10:14:38.993289 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 10:14:38.993289 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 10:14:38.994372 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 10:14:38.994372 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:38.994372 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:38.994372 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:38.994372 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:38.994372 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:38.994372 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:38.994372 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:38.994372 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:38.994372 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:38.994372 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:38.994372 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:38.994372 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:38.994372 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:38.994372 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:38.994372 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:38.994372 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:38.995527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 10:14:38.995527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 10:14:38.995527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 10:14:38.995527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 10:14:38.995527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 10:14:38.995527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 10:14:38.995527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:38.995527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:38.997627 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 471/948] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:14:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:39.003927 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5063a479-e912-43fb-b1e2-52b568d8cd1c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:39.058015 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:39.111632 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5063a479-e912-43fb-b1e2-52b568d8cd1c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:39.148888 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d782d0-73c8-47d7-a8d6-7b0d69883ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:39.150237 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d782d0-73c8-47d7-a8d6-7b0d69883ae1 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:39.150654 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d782d0-73c8-47d7-a8d6-7b0d69883ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:39.152749 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d782d0-73c8-47d7-a8d6-7b0d69883ae1 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:39.153634 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d782d0-73c8-47d7-a8d6-7b0d69883ae1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:39.154099 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d782d0-73c8-47d7-a8d6-7b0d69883ae1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:39.155943 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d782d0-73c8-47d7-a8d6-7b0d69883ae1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:39.177665 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 478/949] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 5 10:14:39 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:39.526107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dba618-85d8-4ae9-a29d-4c7702e069e3 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:39.526651 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dba618-85d8-4ae9-a29d-4c7702e069e3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:39.526651 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dba618-85d8-4ae9-a29d-4c7702e069e3 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:39.528296 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 472/950] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:39.533894 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ef514-9196-44f9-916e-053e78833a8d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:39.534186 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ef514-9196-44f9-916e-053e78833a8d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:39.534458 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ef514-9196-44f9-916e-053e78833a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:39.554094 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e76ef514-9196-44f9-916e-053e78833a8d None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:39.555630 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e76ef514-9196-44f9-916e-053e78833a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:39.622531 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 479/951] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:14:39.645626 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dba618-85d8-4ae9-a29d-4c7702e069e3 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:39.676528 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:39.760453 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dba618-85d8-4ae9-a29d-4c7702e069e3 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:39.799634 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e97cd2b-d826-42e9-bc0f-26191c44c63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:39.800246 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97cd2b-d826-42e9-bc0f-26191c44c63a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:39.800523 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97cd2b-d826-42e9-bc0f-26191c44c63a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:39.800788 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97cd2b-d826-42e9-bc0f-26191c44c63a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:39.811537 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e97cd2b-d826-42e9-bc0f-26191c44c63a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:14:39.816508 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-7e97cd2b-d826-42e9-bc0f-26191c44c63a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 10:14:39.816508 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:39.816508 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:39.816508 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:39.816508 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:39.816508 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:39.816508 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:39.816508 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:39.816508 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:39.816508 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:39.816508 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:39.816508 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:39.816508 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:14:39.816508 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:14:39.816508 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:14:39.816508 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:14:39.817625 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:14:39.817625 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:14:39.817625 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:14:39.817625 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:14:39.817625 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:39.817625 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:39.817625 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:39.817625 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:39.817625 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:39.817625 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:39.817625 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:39.817625 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:39.817625 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:39.817625 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:39.817625 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:39.817625 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:39.818679 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:39.818679 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:39.818679 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:39.818679 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:39.818679 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:14:39.818679 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:14:39.818679 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:14:39.818679 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:14:39.818679 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:14:39.818679 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:14:39.818679 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 10:14:39.818679 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:39.822843 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 473/952] 10.222.0.40 () {56 vars in 1120 bytes} [Fri Aug 5 10:14:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:39.824338 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76ef514-9196-44f9-916e-053e78833a8d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:39.865009 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:39.923337 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76ef514-9196-44f9-916e-053e78833a8d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:39.956949 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3103a71d-e024-4f59-b227-7cc8fa12ff57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:39.957576 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103a71d-e024-4f59-b227-7cc8fa12ff57 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:39.957850 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103a71d-e024-4f59-b227-7cc8fa12ff57 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:39.958118 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103a71d-e024-4f59-b227-7cc8fa12ff57 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:39.958826 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3103a71d-e024-4f59-b227-7cc8fa12ff57 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:39.959157 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3103a71d-e024-4f59-b227-7cc8fa12ff57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:39.960521 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3103a71d-e024-4f59-b227-7cc8fa12ff57 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:39.970611 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 480/953] 10.222.0.40 () {56 vars in 1115 bytes} [Fri Aug 5 10:14:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:39.976170 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e97cd2b-d826-42e9-bc0f-26191c44c63a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:40.061331 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d8813-736e-498c-8e0a-83453c144e42 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:40.061710 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d8813-736e-498c-8e0a-83453c144e42 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:40.062082 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d8813-736e-498c-8e0a-83453c144e42 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:40.063819 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 481/954] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 10:14:40.068637 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af40cb-1f5c-4cc4-a862-34c5daadbee6 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:40.068871 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:40.069092 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af40cb-1f5c-4cc4-a862-34c5daadbee6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:40.069457 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af40cb-1f5c-4cc4-a862-34c5daadbee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:40.081789 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e97cd2b-d826-42e9-bc0f-26191c44c63a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:40.089299 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d7af40cb-1f5c-4cc4-a862-34c5daadbee6 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:40.090592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d7af40cb-1f5c-4cc4-a862-34c5daadbee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:40.116034 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7733d36a-92ea-4861-a321-a1dd480a2d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:40.116678 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7733d36a-92ea-4861-a321-a1dd480a2d50 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:40.117002 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7733d36a-92ea-4861-a321-a1dd480a2d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:40.117273 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7733d36a-92ea-4861-a321-a1dd480a2d50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:40.128376 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7733d36a-92ea-4861-a321-a1dd480a2d50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:40.129923 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7733d36a-92ea-4861-a321-a1dd480a2d50 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:40.144292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-7733d36a-92ea-4861-a321-a1dd480a2d50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:40.144292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:40.144292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:40.144292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:40.144292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:40.144292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:40.144292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:40.144292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:40.144292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:40.144292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:40.144292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:40.144292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:40.144292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:40.144292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:40.144292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:40.146407 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:40.146407 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:40.146407 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:40.146407 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:40.146407 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:40.146407 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:40.146407 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:40.146407 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:40.146407 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:40.146407 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:40.146407 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:40.146407 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:40.146407 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:40.146407 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:40.146407 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:40.146407 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:40.146407 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:40.148623 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:40.148623 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:40.148623 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:40.148623 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:40.152615 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 474/955] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:40.152615 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7733d36a-92ea-4861-a321-a1dd480a2d50 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:40.166994 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 482/956] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 10:14:40.184493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7af40cb-1f5c-4cc4-a862-34c5daadbee6 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:40.219217 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:40.262652 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7733d36a-92ea-4861-a321-a1dd480a2d50 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:40.293535 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7af40cb-1f5c-4cc4-a862-34c5daadbee6 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:40.297209 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3723bdc-5ba8-41ad-b637-9a4210aba872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:40.297362 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3723bdc-5ba8-41ad-b637-9a4210aba872 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:40.297643 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3723bdc-5ba8-41ad-b637-9a4210aba872 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:40.297913 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3723bdc-5ba8-41ad-b637-9a4210aba872 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:40.298606 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3723bdc-5ba8-41ad-b637-9a4210aba872 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:40.298923 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3723bdc-5ba8-41ad-b637-9a4210aba872 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:40.300308 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3723bdc-5ba8-41ad-b637-9a4210aba872 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:40.315524 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:40.317196 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 475/957] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:14:40.318523 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3723bdc-5ba8-41ad-b637-9a4210aba872 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:40.331817 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd91b73-df19-47c3-ac67-a5626f483966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:40.332704 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd91b73-df19-47c3-ac67-a5626f483966 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:40.333121 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd91b73-df19-47c3-ac67-a5626f483966 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:40.333524 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd91b73-df19-47c3-ac67-a5626f483966 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:40.334316 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd91b73-df19-47c3-ac67-a5626f483966 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:40.335848 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd91b73-df19-47c3-ac67-a5626f483966 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:40.345202 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-4fd91b73-df19-47c3-ac67-a5626f483966 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 10:14:40.345202 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:40.345202 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 10:14:40.345202 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 10:14:40.345202 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 10:14:40.345202 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 10:14:40.345202 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 10:14:40.345202 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:40.345202 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 10:14:40.345202 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:40.345202 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 10:14:40.345202 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:40.345202 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:40.345202 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:40.345202 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:40.345202 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:40.345202 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:40.346466 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:40.346466 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:40.346466 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:40.346466 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:40.346466 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:40.346466 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:40.346466 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 10:14:40.346466 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 10:14:40.346466 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 10:14:40.346466 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 10:14:40.346466 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:40.346466 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:40.346466 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:40.346466 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:40.346466 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:40.346466 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:40.347637 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:40.347637 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:40.347637 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:40.347637 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:40.347637 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:40.347637 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:40.347637 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:40.347637 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:40.347637 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:40.347637 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:40.347637 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 10:14:40.347637 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:40.347637 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 10:14:40.347637 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:40.354997 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 483/958] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:14:40 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:14:40.356113 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd91b73-df19-47c3-ac67-a5626f483966 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:40.428027 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3723bdc-5ba8-41ad-b637-9a4210aba872 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:40.464504 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a7acb0-4149-4916-bc1b-0ba122c36b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:40.465604 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a7acb0-4149-4916-bc1b-0ba122c36b68 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:40.466077 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a7acb0-4149-4916-bc1b-0ba122c36b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:40.466537 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a7acb0-4149-4916-bc1b-0ba122c36b68 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:40.474541 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a7acb0-4149-4916-bc1b-0ba122c36b68 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:40.476425 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a7acb0-4149-4916-bc1b-0ba122c36b68 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:40.479792 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:40.485859 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd91b73-df19-47c3-ac67-a5626f483966 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:40.490619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-78a7acb0-4149-4916-bc1b-0ba122c36b68 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 10:14:40.490619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:40.490619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:40.490619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:40.490619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:40.490619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:40.490619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:40.490619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:40.490619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:40.490619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:40.490619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:40.490619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:40.490619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 10:14:40.490619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 10:14:40.490619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 10:14:40.490619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 10:14:40.491585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:40.491585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:40.491585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 10:14:40.491585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:40.491585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 10:14:40.491585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:40.491585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:40.491585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:40.491585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:40.491585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:40.491585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:40.491585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:40.491585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:40.491585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:40.491585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:40.491585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:40.491585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:40.491585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:40.492509 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:40.492509 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:40.492509 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 10:14:40.492509 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 10:14:40.492509 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 10:14:40.492509 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 10:14:40.492509 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 10:14:40.492509 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 10:14:40.492509 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 10:14:40.492509 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:40.495086 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 476/959] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Aug 5 10:14:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:40.501667 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a7acb0-4149-4916-bc1b-0ba122c36b68 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:40.526063 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee1c120-fe2b-4267-904d-e949a8561e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:40.526833 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1c120-fe2b-4267-904d-e949a8561e51 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:40.527144 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1c120-fe2b-4267-904d-e949a8561e51 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:40.527415 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1c120-fe2b-4267-904d-e949a8561e51 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:40.528131 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee1c120-fe2b-4267-904d-e949a8561e51 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:40.528395 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee1c120-fe2b-4267-904d-e949a8561e51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:40.530000 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee1c120-fe2b-4267-904d-e949a8561e51 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:40.540146 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 484/960] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 5 10:14:40 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:40.541215 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee1c120-fe2b-4267-904d-e949a8561e51 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:40.580308 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:40.624670 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a7acb0-4149-4916-bc1b-0ba122c36b68 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:40.660498 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c58acd1-2ccf-4a94-908d-fab94a820a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:40.661281 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58acd1-2ccf-4a94-908d-fab94a820a74 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:40.661677 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58acd1-2ccf-4a94-908d-fab94a820a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:40.662043 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58acd1-2ccf-4a94-908d-fab94a820a74 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:40.663087 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c58acd1-2ccf-4a94-908d-fab94a820a74 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:40.663393 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c58acd1-2ccf-4a94-908d-fab94a820a74 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:40.665004 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c58acd1-2ccf-4a94-908d-fab94a820a74 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:40.674885 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee1c120-fe2b-4267-904d-e949a8561e51 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:40.688953 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 477/961] 10.222.0.40 () {56 vars in 1149 bytes} [Fri Aug 5 10:14:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:14:40.690313 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4172ed-ac1a-4c14-a023-d2d75da532dc None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:40.690606 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4172ed-ac1a-4c14-a023-d2d75da532dc None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:40.690883 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4172ed-ac1a-4c14-a023-d2d75da532dc None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:40.692429 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 478/962] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 10:14:40.693128 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4172ed-ac1a-4c14-a023-d2d75da532dc None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:40.716331 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bd0c72-eea9-4669-9263-3ac69be8382d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:40.716954 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd0c72-eea9-4669-9263-3ac69be8382d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:40.717231 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd0c72-eea9-4669-9263-3ac69be8382d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:40.717499 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd0c72-eea9-4669-9263-3ac69be8382d None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:40.718207 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bd0c72-eea9-4669-9263-3ac69be8382d None admin] RBAC: Adding query filter params (type=compute) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:40.718470 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bd0c72-eea9-4669-9263-3ac69be8382d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:40.719877 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bd0c72-eea9-4669-9263-3ac69be8382d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:40.725488 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 485/963] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 5 10:14:40 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:14:40.727585 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103c969-93fa-4354-a453-0dde0a69738f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:40.728094 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103c969-93fa-4354-a453-0dde0a69738f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:40.728541 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103c969-93fa-4354-a453-0dde0a69738f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:40.751282 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-5103c969-93fa-4354-a453-0dde0a69738f None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:40.752682 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5103c969-93fa-4354-a453-0dde0a69738f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:40.775450 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:40.839057 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 486/964] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:14:40.854168 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4172ed-ac1a-4c14-a023-d2d75da532dc None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:40.865242 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5103c969-93fa-4354-a453-0dde0a69738f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:40.872159 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:40.892317 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef52659-e43d-4e03-b109-ac414f7160f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:40.892959 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef52659-e43d-4e03-b109-ac414f7160f3 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:40.893240 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef52659-e43d-4e03-b109-ac414f7160f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:40.893516 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef52659-e43d-4e03-b109-ac414f7160f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:40.973385 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef52659-e43d-4e03-b109-ac414f7160f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:40.974838 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef52659-e43d-4e03-b109-ac414f7160f3 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:40.992404 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5103c969-93fa-4354-a453-0dde0a69738f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:41.027111 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcc0d8e-b272-4022-9f03-329dab002f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:41.028163 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc0d8e-b272-4022-9f03-329dab002f21 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:41.028637 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc0d8e-b272-4022-9f03-329dab002f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:41.029100 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc0d8e-b272-4022-9f03-329dab002f21 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:41.030020 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcc0d8e-b272-4022-9f03-329dab002f21 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:41.031641 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcc0d8e-b272-4022-9f03-329dab002f21 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:41.040761 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-2fcc0d8e-b272-4022-9f03-329dab002f21 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 10:14:41.040761 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:41.040761 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 10:14:41.040761 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 10:14:41.040761 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 10:14:41.040761 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 10:14:41.040761 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 10:14:41.040761 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:41.040761 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 10:14:41.040761 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:41.040761 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 10:14:41.040761 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:41.040761 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:41.040761 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:41.040761 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:41.040761 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:41.040761 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:41.042383 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:41.042383 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:41.042383 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:41.042383 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:41.042383 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:41.042383 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:41.042383 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 10:14:41.042383 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 10:14:41.042383 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 10:14:41.042383 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 10:14:41.042383 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:41.042383 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:41.042383 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:41.042383 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:41.042383 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:41.042383 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:41.043635 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:41.043635 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:41.043635 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:41.043635 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:41.043635 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:41.043635 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:41.043635 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:41.043635 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:41.043635 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:41.043635 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:41.043635 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 10:14:41.043635 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:41.043635 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 10:14:41.043635 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:41.046158 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 487/965] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Aug 5 10:14:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:14:41.052434 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcc0d8e-b272-4022-9f03-329dab002f21 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:41.085083 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 479/966] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:14:40 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 10:14:41.086414 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef52659-e43d-4e03-b109-ac414f7160f3 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:41.087336 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:41.105931 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:41.181818 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcc0d8e-b272-4022-9f03-329dab002f21 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:41.201513 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef52659-e43d-4e03-b109-ac414f7160f3 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:41.216936 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaa5c54-d28a-4a8b-b4a0-6fb37287108a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:41.217700 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa5c54-d28a-4a8b-b4a0-6fb37287108a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:41.218951 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa5c54-d28a-4a8b-b4a0-6fb37287108a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:41.218951 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa5c54-d28a-4a8b-b4a0-6fb37287108a None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:41.219283 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaa5c54-d28a-4a8b-b4a0-6fb37287108a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:41.219587 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaa5c54-d28a-4a8b-b4a0-6fb37287108a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:41.220962 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaa5c54-d28a-4a8b-b4a0-6fb37287108a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:41.226097 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 488/967] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Aug 5 10:14:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:14:41.231882 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaa5c54-d28a-4a8b-b4a0-6fb37287108a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:41.250314 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581216e0-531f-46e7-a1bd-785dc3a2fb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:41.251359 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581216e0-531f-46e7-a1bd-785dc3a2fb29 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:41.252052 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581216e0-531f-46e7-a1bd-785dc3a2fb29 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:41.252492 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581216e0-531f-46e7-a1bd-785dc3a2fb29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:41.266312 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581216e0-531f-46e7-a1bd-785dc3a2fb29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:41.268159 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581216e0-531f-46e7-a1bd-785dc3a2fb29 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:41.287353 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-581216e0-531f-46e7-a1bd-785dc3a2fb29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:41.287353 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:41.287353 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:41.287353 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:41.287353 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:41.287353 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:41.287353 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:41.287353 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:41.287353 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:41.287353 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:41.287353 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:41.287353 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:41.287353 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:41.287353 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:41.287353 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:41.288462 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:41.288462 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:41.288462 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:41.288462 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:41.288462 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:41.288462 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:41.288462 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:41.288462 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:41.288462 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:41.288462 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:41.288462 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:41.288462 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:41.288462 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:41.288462 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:41.288462 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:41.288462 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:41.288462 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:41.289353 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:41.289353 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:41.289353 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:41.289353 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:41.291792 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 480/968] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:41.297354 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b86125-16cf-4169-9b04-8ede9725b7c5 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:41.297643 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b86125-16cf-4169-9b04-8ede9725b7c5 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:41.297918 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b86125-16cf-4169-9b04-8ede9725b7c5 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:41.299391 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 481/969] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 10:14:41.303525 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b86125-16cf-4169-9b04-8ede9725b7c5 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:41.325964 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:41.345856 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaa5c54-d28a-4a8b-b4a0-6fb37287108a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:41.376172 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:41.380355 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a391c7-cf63-41dd-b27f-bb47588434e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:41.380995 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a391c7-cf63-41dd-b27f-bb47588434e7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:41.381273 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a391c7-cf63-41dd-b27f-bb47588434e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:41.381545 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a391c7-cf63-41dd-b27f-bb47588434e7 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:41.382247 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a391c7-cf63-41dd-b27f-bb47588434e7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:41.382549 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a391c7-cf63-41dd-b27f-bb47588434e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:41.384365 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a391c7-cf63-41dd-b27f-bb47588434e7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:41.390186 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 489/970] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Aug 5 10:14:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:14:41.391284 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85694aa9-9fad-4126-8633-38933a520dd5 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:41.391567 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85694aa9-9fad-4126-8633-38933a520dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:41.391838 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85694aa9-9fad-4126-8633-38933a520dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:41.410114 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-85694aa9-9fad-4126-8633-38933a520dd5 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:41.411322 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-85694aa9-9fad-4126-8633-38933a520dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:41.414712 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b86125-16cf-4169-9b04-8ede9725b7c5 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:41.451335 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f69ebc3-0603-4972-98b5-d029109c3f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:41.451953 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69ebc3-0603-4972-98b5-d029109c3f8f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:41.452211 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69ebc3-0603-4972-98b5-d029109c3f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:41.452502 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69ebc3-0603-4972-98b5-d029109c3f8f None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:41.453278 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f69ebc3-0603-4972-98b5-d029109c3f8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:41.453585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f69ebc3-0603-4972-98b5-d029109c3f8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:41.455048 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f69ebc3-0603-4972-98b5-d029109c3f8f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:41.472233 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 482/971] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:41.472233 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f69ebc3-0603-4972-98b5-d029109c3f8f None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:41.487878 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 490/972] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:14:41.488733 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85694aa9-9fad-4126-8633-38933a520dd5 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:41.572598 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:41.584416 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f69ebc3-0603-4972-98b5-d029109c3f8f None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:41.595976 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85694aa9-9fad-4126-8633-38933a520dd5 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:41.616083 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:41.623479 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20906352-8b7d-4f74-a778-99c3b0459011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:41.624099 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20906352-8b7d-4f74-a778-99c3b0459011 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:41.624522 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20906352-8b7d-4f74-a778-99c3b0459011 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:41.624649 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20906352-8b7d-4f74-a778-99c3b0459011 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:41.625327 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20906352-8b7d-4f74-a778-99c3b0459011 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:41.626748 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20906352-8b7d-4f74-a778-99c3b0459011 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:41.632814 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c13f21-1e4b-46d8-9b0d-0c5381ba8cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:41.633457 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c13f21-1e4b-46d8-9b0d-0c5381ba8cd5 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:41.633734 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c13f21-1e4b-46d8-9b0d-0c5381ba8cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:41.634001 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c13f21-1e4b-46d8-9b0d-0c5381ba8cd5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:41.640903 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c13f21-1e4b-46d8-9b0d-0c5381ba8cd5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:41.642454 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c13f21-1e4b-46d8-9b0d-0c5381ba8cd5 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:41.650382 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 483/973] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:14:41 2022] POST /identity/v3/endpoints => generated 349 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:14:41.651090 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20906352-8b7d-4f74-a778-99c3b0459011 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:41.657047 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-a8c13f21-1e4b-46d8-9b0d-0c5381ba8cd5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:41.657047 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:41.657047 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:41.657047 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:41.657047 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:41.657047 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:41.657047 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:41.657047 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:41.657047 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:41.657047 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:41.657047 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:41.657047 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:41.657047 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 10:14:41.657047 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 10:14:41.657047 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 10:14:41.658042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 10:14:41.658042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:41.658042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:41.658042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:41.658042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:41.658042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:41.658042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:41.658042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:41.658042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:41.658042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:41.658042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:41.658042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:41.658042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:41.658042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:41.658042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:41.658042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:41.658042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:41.658982 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 10:14:41.658982 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 10:14:41.658982 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 10:14:41.658982 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 10:14:41.658982 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 10:14:41.658982 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 10:14:41.658982 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:41.658982 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:41.658982 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 491/974] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:14:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:41.676729 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c13f21-1e4b-46d8-9b0d-0c5381ba8cd5 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:41.795035 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c13f21-1e4b-46d8-9b0d-0c5381ba8cd5 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:41.804437 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20906352-8b7d-4f74-a778-99c3b0459011 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:41.841177 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75c4ce-eb39-4ef6-b074-eebde21efff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:41.841810 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75c4ce-eb39-4ef6-b074-eebde21efff7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:41.842093 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75c4ce-eb39-4ef6-b074-eebde21efff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:41.842366 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75c4ce-eb39-4ef6-b074-eebde21efff7 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:41.843176 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f75c4ce-eb39-4ef6-b074-eebde21efff7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:41.843455 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f75c4ce-eb39-4ef6-b074-eebde21efff7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:41.844960 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f75c4ce-eb39-4ef6-b074-eebde21efff7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:41.847604 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:41.851866 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e7abcf-0b7e-40e3-9860-76dc0b8eb653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:41.852504 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7abcf-0b7e-40e3-9860-76dc0b8eb653 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:41.852868 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7abcf-0b7e-40e3-9860-76dc0b8eb653 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:41.853514 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7abcf-0b7e-40e3-9860-76dc0b8eb653 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:41.853941 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e7abcf-0b7e-40e3-9860-76dc0b8eb653 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:41.855476 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e7abcf-0b7e-40e3-9860-76dc0b8eb653 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:41.858064 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: WARNING keystone.server.flask.common [None req-d6e7abcf-0b7e-40e3-9860-76dc0b8eb653 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 05 10:14:41.860117 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 492/975] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 10:14:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:14:41.866248 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75c4ce-eb39-4ef6-b074-eebde21efff7 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:41.877273 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.sql.core [None req-d6e7abcf-0b7e-40e3-9860-76dc0b8eb653 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 05 10:14:41.877273 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 05 10:14:41.877273 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [parameters: {'user_id': 'db495902585943fd869c39416ed71240', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 05 10:14:41.877273 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80141) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 05 10:14:41.895904 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-d6e7abcf-0b7e-40e3-9860-76dc0b8eb653 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 05 10:14:41.895904 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:41.895904 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 05 10:14:41.895904 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 05 10:14:41.895904 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 05 10:14:41.895904 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 05 10:14:41.895904 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 05 10:14:41.895904 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application result = self._query(query) Aug 05 10:14:41.895904 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 05 10:14:41.895904 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application conn.query(q) Aug 05 10:14:41.895904 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 05 10:14:41.895904 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 05 10:14:41.895904 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 05 10:14:41.895904 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application result.read() Aug 05 10:14:41.895904 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 05 10:14:41.896911 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 05 10:14:41.896911 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 05 10:14:41.896911 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 05 10:14:41.896911 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 05 10:14:41.896911 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 05 10:14:41.896911 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 05 10:14:41.896911 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 05 10:14:41.896911 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 05 10:14:41.896911 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:41.896911 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 05 10:14:41.896911 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:41.896911 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:41.896911 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 05 10:14:41.896911 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 05 10:14:41.896911 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 05 10:14:41.896911 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 05 10:14:41.896911 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 05 10:14:41.896911 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application next(self.gen) Aug 05 10:14:41.897891 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 05 10:14:41.897891 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application yield resource Aug 05 10:14:41.897891 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 05 10:14:41.897891 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application next(self.gen) Aug 05 10:14:41.897891 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 05 10:14:41.897891 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self.session.rollback() Aug 05 10:14:41.897891 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 05 10:14:41.897891 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self.force_reraise() Aug 05 10:14:41.897891 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 05 10:14:41.897891 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise self.value Aug 05 10:14:41.897891 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 05 10:14:41.897891 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 05 10:14:41.897891 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 05 10:14:41.897891 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application session.commit() Aug 05 10:14:41.897891 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 05 10:14:41.897891 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 05 10:14:41.897891 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 05 10:14:41.898780 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._prepare_impl() Aug 05 10:14:41.898780 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 05 10:14:41.898780 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self.session.flush() Aug 05 10:14:41.898780 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 05 10:14:41.898780 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._flush(objects) Aug 05 10:14:41.898780 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 05 10:14:41.898780 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 05 10:14:41.898780 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 05 10:14:41.898780 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application compat.raise_( Aug 05 10:14:41.898780 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 05 10:14:41.898780 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception Aug 05 10:14:41.898780 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 05 10:14:41.898780 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application flush_context.execute() Aug 05 10:14:41.898780 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 05 10:14:41.898780 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rec.execute(self) Aug 05 10:14:41.898780 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 05 10:14:41.898780 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 05 10:14:41.900107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 05 10:14:41.900107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 05 10:14:41.900107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 05 10:14:41.900107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 05 10:14:41.900107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 05 10:14:41.900107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 05 10:14:41.900107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 05 10:14:41.900107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 05 10:14:41.900107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 05 10:14:41.900107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 05 10:14:41.900107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 05 10:14:41.900107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 05 10:14:41.900107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 05 10:14:41.900107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 05 10:14:41.900107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 05 10:14:41.900107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception Aug 05 10:14:41.901505 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 05 10:14:41.901505 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 05 10:14:41.901505 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 05 10:14:41.901505 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 05 10:14:41.901505 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 05 10:14:41.901505 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application result = self._query(query) Aug 05 10:14:41.901505 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 05 10:14:41.901505 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application conn.query(q) Aug 05 10:14:41.901505 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 05 10:14:41.901505 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 05 10:14:41.901505 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 05 10:14:41.901505 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application result.read() Aug 05 10:14:41.901505 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 05 10:14:41.901505 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 05 10:14:41.901505 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 05 10:14:41.901505 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 05 10:14:41.902564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 05 10:14:41.902564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 05 10:14:41.902564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 05 10:14:41.902564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 05 10:14:41.902564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 05 10:14:41.902564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 05 10:14:41.902564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [parameters: {'user_id': 'db495902585943fd869c39416ed71240', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 05 10:14:41.902564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 05 10:14:41.902564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:41.902564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 10:14:41.902564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:41.902564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:41.902564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:41.902564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:41.902564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:41.903627 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:41.903627 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:41.903627 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:41.903627 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:41.903627 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:41.903627 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:41.903627 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:41.903627 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 05 10:14:41.903627 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 05 10:14:41.903627 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:41.903627 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:41.903627 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 10:14:41.903627 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:41.903627 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 10:14:41.903627 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:41.903627 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 05 10:14:41.903627 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 05 10:14:41.904873 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 05 10:14:41.904873 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 05 10:14:41.904873 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 05 10:14:41.904873 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 05 10:14:41.904873 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 05 10:14:41.904873 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:41.904873 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 484/976] 10.222.0.40 () {60 vars in 1131 bytes} [Fri Aug 5 10:14:41 2022] POST /identity/v3/users => generated 160 bytes in 247 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:41.918924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e7abcf-0b7e-40e3-9860-76dc0b8eb653 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:41.991918 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:41.997573 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75c4ce-eb39-4ef6-b074-eebde21efff7 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:42.034697 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e7abcf-0b7e-40e3-9860-76dc0b8eb653 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:42.041475 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b865d-691d-47f1-b99d-bff960a149dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:42.042925 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b865d-691d-47f1-b99d-bff960a149dc None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:42.043869 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b865d-691d-47f1-b99d-bff960a149dc None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:42.044347 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b865d-691d-47f1-b99d-bff960a149dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:42.045347 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956b865d-691d-47f1-b99d-bff960a149dc None admin] RBAC: Adding query filter params (role.id=f10badb1efd340eda8d6358dc6bd9c28, scope.project.id=36b8b505012d4d0bac54845b6799e743, user.id=52fc3ba90dc045c2a5b7bb730da96acc) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:42.046182 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956b865d-691d-47f1-b99d-bff960a149dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:42.048420 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956b865d-691d-47f1-b99d-bff960a149dc None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:42.063503 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 493/977] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:14:41 2022] GET /identity/v3/role_assignments?user.id=52fc3ba90dc045c2a5b7bb730da96acc&scope.project.id=36b8b505012d4d0bac54845b6799e743&role.id=f10badb1efd340eda8d6358dc6bd9c28 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:14:42.078741 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f93af-10aa-4b10-bb33-4e9690147e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:42.079456 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f93af-10aa-4b10-bb33-4e9690147e54 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:42.079738 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f93af-10aa-4b10-bb33-4e9690147e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:42.080014 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f93af-10aa-4b10-bb33-4e9690147e54 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:42.089262 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02f93af-10aa-4b10-bb33-4e9690147e54 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:42.091009 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02f93af-10aa-4b10-bb33-4e9690147e54 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:42.104243 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-f02f93af-10aa-4b10-bb33-4e9690147e54 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 10:14:42.104243 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:42.104243 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:42.104243 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:42.104243 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:42.104243 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:42.104243 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:42.104243 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:42.104243 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:42.104243 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:42.104243 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:42.104243 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:42.104243 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 10:14:42.104243 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 10:14:42.104243 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 10:14:42.104243 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 10:14:42.106108 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:42.106108 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:42.106108 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 10:14:42.106108 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:42.106108 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 10:14:42.106108 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:42.106108 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:42.106108 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:42.106108 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:42.106108 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:42.106108 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:42.106108 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:42.106108 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:42.106108 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:42.106108 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:42.106108 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:42.106108 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:42.106108 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:42.107404 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:42.107404 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:42.107404 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 10:14:42.107404 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 10:14:42.107404 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 10:14:42.107404 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 10:14:42.107404 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 10:14:42.107404 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 10:14:42.107404 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 10:14:42.107404 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:42.107404 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 485/978] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:14:41 2022] GET /identity/v3/users/cinder => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:42.112067 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b865d-691d-47f1-b99d-bff960a149dc None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:42.188709 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:42.234932 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b865d-691d-47f1-b99d-bff960a149dc None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:42.275768 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f104f-fde2-495e-a541-788952b31e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:42.276959 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f104f-fde2-495e-a541-788952b31e1f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:42.277445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f104f-fde2-495e-a541-788952b31e1f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:42.277916 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f104f-fde2-495e-a541-788952b31e1f None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:42.278873 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30f104f-fde2-495e-a541-788952b31e1f None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:42.279408 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30f104f-fde2-495e-a541-788952b31e1f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:42.281058 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30f104f-fde2-495e-a541-788952b31e1f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:42.317936 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 494/979] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 10:14:42 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:42.797419 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f3c95-c3fb-48fc-babd-eed2ac64a1f3 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:42.797971 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f3c95-c3fb-48fc-babd-eed2ac64a1f3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:42.797971 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f3c95-c3fb-48fc-babd-eed2ac64a1f3 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:42.802301 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 486/980] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:42.807952 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6a073-5503-482b-b3d7-45cdb3b21b41 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:42.808210 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6a073-5503-482b-b3d7-45cdb3b21b41 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:42.808521 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6a073-5503-482b-b3d7-45cdb3b21b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:42.828053 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d0c6a073-5503-482b-b3d7-45cdb3b21b41 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:42.829291 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d0c6a073-5503-482b-b3d7-45cdb3b21b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:42.902777 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 495/981] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:14:42.925561 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f3c95-c3fb-48fc-babd-eed2ac64a1f3 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:43.026675 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f3c95-c3fb-48fc-babd-eed2ac64a1f3 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:43.029383 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:43.062117 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf5682c-a724-4866-9274-ca7b5c1300a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:43.062673 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf5682c-a724-4866-9274-ca7b5c1300a8 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:43.062983 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf5682c-a724-4866-9274-ca7b5c1300a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:43.063259 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf5682c-a724-4866-9274-ca7b5c1300a8 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:43.063965 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf5682c-a724-4866-9274-ca7b5c1300a8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:43.065341 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf5682c-a724-4866-9274-ca7b5c1300a8 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:43.083649 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 487/982] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 5 10:14:42 2022] POST /identity/v3/services => generated 242 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:14:43.744333 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d220f968-d6e6-4c28-a37e-27fe6c6de69a None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:43.744914 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d220f968-d6e6-4c28-a37e-27fe6c6de69a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:43.744993 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d220f968-d6e6-4c28-a37e-27fe6c6de69a None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:43.746533 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 496/983] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:43.751686 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79510e7f-4b39-4845-aee5-86a6dbc3aaea None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:43.751976 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79510e7f-4b39-4845-aee5-86a6dbc3aaea None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:43.752231 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79510e7f-4b39-4845-aee5-86a6dbc3aaea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:43.772129 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-79510e7f-4b39-4845-aee5-86a6dbc3aaea None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:43.773654 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-79510e7f-4b39-4845-aee5-86a6dbc3aaea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:43.852744 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 488/984] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:43 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:14:43.880898 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d220f968-d6e6-4c28-a37e-27fe6c6de69a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:43.933453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:43.985604 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d220f968-d6e6-4c28-a37e-27fe6c6de69a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:44.021417 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca217af-2749-4ddf-b4c0-205c47a79f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:44.021980 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca217af-2749-4ddf-b4c0-205c47a79f56 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:44.022262 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca217af-2749-4ddf-b4c0-205c47a79f56 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:44.022533 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca217af-2749-4ddf-b4c0-205c47a79f56 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:44.023198 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca217af-2749-4ddf-b4c0-205c47a79f56 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:44.024584 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca217af-2749-4ddf-b4c0-205c47a79f56 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:44.033636 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-aca217af-2749-4ddf-b4c0-205c47a79f56 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 10:14:44.033636 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:44.033636 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 10:14:44.033636 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 10:14:44.033636 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 10:14:44.033636 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 10:14:44.033636 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 10:14:44.033636 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:44.033636 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 10:14:44.033636 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:44.033636 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 10:14:44.033636 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:44.033636 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:44.033636 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:44.033636 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:44.033636 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:44.033636 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:44.034669 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:44.034669 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:44.034669 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:44.034669 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:44.034669 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:44.034669 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:44.034669 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 10:14:44.034669 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 10:14:44.034669 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 10:14:44.034669 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 10:14:44.034669 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:44.034669 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:44.034669 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:44.034669 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:44.034669 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:44.034669 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:44.035879 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:44.035879 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:44.035879 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:44.035879 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:44.035879 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:44.035879 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:44.035879 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:44.035879 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:44.035879 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:44.035879 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:44.035879 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 10:14:44.035879 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:44.035879 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 10:14:44.035879 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:44.039249 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 497/985] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Aug 5 10:14:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:44.040777 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79510e7f-4b39-4845-aee5-86a6dbc3aaea None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:44.130563 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:44.142402 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2ea9f-0aea-4b08-80b4-6f4d81a602eb None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:44.142602 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79510e7f-4b39-4845-aee5-86a6dbc3aaea None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:44.143140 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2ea9f-0aea-4b08-80b4-6f4d81a602eb None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:44.143611 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2ea9f-0aea-4b08-80b4-6f4d81a602eb None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:44.145191 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 498/986] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:44.150041 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4f0f1-fa73-422c-8b65-250137923e36 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:44.150307 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4f0f1-fa73-422c-8b65-250137923e36 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:44.150631 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4f0f1-fa73-422c-8b65-250137923e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:44.171857 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-2db4f0f1-fa73-422c-8b65-250137923e36 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:44.173153 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2db4f0f1-fa73-422c-8b65-250137923e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:44.193599 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b38087-f5f8-49c4-8c0e-db08f0858948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:44.194246 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b38087-f5f8-49c4-8c0e-db08f0858948 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:44.194515 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b38087-f5f8-49c4-8c0e-db08f0858948 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:44.194785 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b38087-f5f8-49c4-8c0e-db08f0858948 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:44.195519 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b38087-f5f8-49c4-8c0e-db08f0858948 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:44.195820 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b38087-f5f8-49c4-8c0e-db08f0858948 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:44.197220 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b38087-f5f8-49c4-8c0e-db08f0858948 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:44.203644 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 489/987] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Aug 5 10:14:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:44.209231 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b38087-f5f8-49c4-8c0e-db08f0858948 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:44.250585 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 499/988] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:44 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 10:14:44.272450 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db4f0f1-fa73-422c-8b65-250137923e36 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:44.314098 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b38087-f5f8-49c4-8c0e-db08f0858948 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:44.315933 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:44.357620 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274ae1a5-576c-4242-8de7-efba65197f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:44.358238 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ae1a5-576c-4242-8de7-efba65197f32 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:44.358765 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ae1a5-576c-4242-8de7-efba65197f32 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:44.359046 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ae1a5-576c-4242-8de7-efba65197f32 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:44.360222 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274ae1a5-576c-4242-8de7-efba65197f32 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:44.360532 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274ae1a5-576c-4242-8de7-efba65197f32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:44.362412 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274ae1a5-576c-4242-8de7-efba65197f32 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:44.368697 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 490/989] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Aug 5 10:14:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:44.375243 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274ae1a5-576c-4242-8de7-efba65197f32 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:44.381361 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db4f0f1-fa73-422c-8b65-250137923e36 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:44.396650 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:44.419006 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f4512-687c-48c4-b52d-27c43f0ba279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:44.419659 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f4512-687c-48c4-b52d-27c43f0ba279 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:44.419938 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f4512-687c-48c4-b52d-27c43f0ba279 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:44.420210 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f4512-687c-48c4-b52d-27c43f0ba279 None admin] PATH_INFO: `/v3/services/image` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:44.420837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305f4512-687c-48c4-b52d-27c43f0ba279 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:44.422214 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305f4512-687c-48c4-b52d-27c43f0ba279 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:44.431490 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-305f4512-687c-48c4-b52d-27c43f0ba279 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 10:14:44.431490 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:44.431490 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 10:14:44.431490 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 10:14:44.431490 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 10:14:44.431490 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 10:14:44.431490 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 10:14:44.431490 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:44.431490 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 10:14:44.431490 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:44.431490 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 10:14:44.431490 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:44.431490 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:44.431490 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:44.431490 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:44.431490 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:44.431490 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:44.433084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:44.433084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:44.433084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:44.433084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:44.433084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:44.433084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:44.433084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 10:14:44.433084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 10:14:44.433084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 10:14:44.433084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 10:14:44.433084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:44.433084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:44.433084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:44.433084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:44.433084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:44.433084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:44.433932 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:44.433932 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:44.433932 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:44.433932 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:44.433932 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:44.433932 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:44.433932 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:44.433932 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:44.433932 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:44.433932 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:44.433932 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 10:14:44.433932 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:44.433932 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 10:14:44.433932 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:44.437652 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 500/990] 10.222.0.40 () {56 vars in 1105 bytes} [Fri Aug 5 10:14:44 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:44.438459 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f4512-687c-48c4-b52d-27c43f0ba279 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:44.490051 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274ae1a5-576c-4242-8de7-efba65197f32 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:44.527030 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbff800-44da-45a6-aca6-9831e10cf6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:44.527730 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbff800-44da-45a6-aca6-9831e10cf6ec None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:44.528015 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbff800-44da-45a6-aca6-9831e10cf6ec None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:44.528334 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbff800-44da-45a6-aca6-9831e10cf6ec None admin] PATH_INFO: `/v3/endpoints` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:44.529112 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbff800-44da-45a6-aca6-9831e10cf6ec None admin] RBAC: Adding query filter params (interface=admin, service_id=4c5766718189433a82fa0f7d7a84143f, region_id=RegionOne) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:44.529419 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbff800-44da-45a6-aca6-9831e10cf6ec None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:44.530818 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbff800-44da-45a6-aca6-9831e10cf6ec None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:44.536376 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 491/991] 10.222.0.40 () {56 vars in 1249 bytes} [Fri Aug 5 10:14:44 2022] GET /identity/v3/endpoints?service_id=4c5766718189433a82fa0f7d7a84143f&interface=admin®ion_id=RegionOne => generated 193 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:44.542703 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbff800-44da-45a6-aca6-9831e10cf6ec None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:44.547043 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f4512-687c-48c4-b52d-27c43f0ba279 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:44.555070 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:44.583714 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a802379d-2751-494a-8558-b9948d41a74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:44.584345 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802379d-2751-494a-8558-b9948d41a74d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:44.584625 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802379d-2751-494a-8558-b9948d41a74d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:44.584898 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802379d-2751-494a-8558-b9948d41a74d None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:44.585580 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a802379d-2751-494a-8558-b9948d41a74d None admin] RBAC: Adding query filter params (name=image) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:44.585889 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a802379d-2751-494a-8558-b9948d41a74d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:44.587291 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a802379d-2751-494a-8558-b9948d41a74d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:44.592289 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 501/992] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:44 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:14:44.597888 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a802379d-2751-494a-8558-b9948d41a74d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:44.634353 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:44.654824 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbff800-44da-45a6-aca6-9831e10cf6ec None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:44.691527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c773eb4-40a8-486a-bd0d-5cdea640e874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:44.692157 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c773eb4-40a8-486a-bd0d-5cdea640e874 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:44.692441 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c773eb4-40a8-486a-bd0d-5cdea640e874 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:44.692715 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c773eb4-40a8-486a-bd0d-5cdea640e874 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:44.693373 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c773eb4-40a8-486a-bd0d-5cdea640e874 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:44.694742 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c773eb4-40a8-486a-bd0d-5cdea640e874 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:44.710950 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 492/993] 10.222.0.40 () {56 vars in 1087 bytes} [Fri Aug 5 10:14:44 2022] GET /identity/v3/services => generated 1727 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 10:14:44.711176 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a802379d-2751-494a-8558-b9948d41a74d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:44.748980 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998ecf44-236f-46b6-a723-b2b9f4db9054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:44.749658 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998ecf44-236f-46b6-a723-b2b9f4db9054 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:44.750017 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998ecf44-236f-46b6-a723-b2b9f4db9054 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:44.750359 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998ecf44-236f-46b6-a723-b2b9f4db9054 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:44.751155 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998ecf44-236f-46b6-a723-b2b9f4db9054 None admin] RBAC: Adding query filter params (type=image) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:44.751525 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998ecf44-236f-46b6-a723-b2b9f4db9054 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:44.753028 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998ecf44-236f-46b6-a723-b2b9f4db9054 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:44.763530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 502/994] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:44 2022] GET /identity/v3/services?type=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:14:45.069657 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7b379-3dbf-4c2a-93e2-e20e981b7a45 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:45.070212 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7b379-3dbf-4c2a-93e2-e20e981b7a45 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:45.070212 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7b379-3dbf-4c2a-93e2-e20e981b7a45 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:45.071752 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 493/995] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:45.077376 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc60d3-9ab2-4cc6-911a-62c20f507fef None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:45.077653 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc60d3-9ab2-4cc6-911a-62c20f507fef None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:45.077924 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc60d3-9ab2-4cc6-911a-62c20f507fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:45.096762 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-28dc60d3-9ab2-4cc6-911a-62c20f507fef None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:45.097931 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-28dc60d3-9ab2-4cc6-911a-62c20f507fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:45.126319 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:45.177363 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 503/996] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:45 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:14:45.199439 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae7b379-3dbf-4c2a-93e2-e20e981b7a45 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:45.308501 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae7b379-3dbf-4c2a-93e2-e20e981b7a45 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:45.342572 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc2bfe-2a18-4e12-a3ac-4c43ad1bc6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:45.343162 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc2bfe-2a18-4e12-a3ac-4c43ad1bc6cb None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:45.343443 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc2bfe-2a18-4e12-a3ac-4c43ad1bc6cb None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:45.343716 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc2bfe-2a18-4e12-a3ac-4c43ad1bc6cb None admin] PATH_INFO: `/v3/services/compute` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:45.344338 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc2bfe-2a18-4e12-a3ac-4c43ad1bc6cb None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:45.345733 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc2bfe-2a18-4e12-a3ac-4c43ad1bc6cb None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:45.354947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-5ddc2bfe-2a18-4e12-a3ac-4c43ad1bc6cb None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 10:14:45.354947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:45.354947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 10:14:45.354947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 10:14:45.354947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 10:14:45.354947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 10:14:45.354947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 10:14:45.354947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:45.354947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 10:14:45.354947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:45.354947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 10:14:45.354947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:45.354947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:45.354947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:45.354947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:45.354947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:45.354947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:45.355952 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:45.355952 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:45.355952 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:45.355952 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:45.355952 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:45.355952 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:45.355952 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 10:14:45.355952 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 10:14:45.355952 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 10:14:45.355952 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 10:14:45.355952 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:45.355952 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:45.355952 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:45.355952 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:45.355952 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:45.355952 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:45.361853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:45.361853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:45.361853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:45.361853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:45.361853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:45.361853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:45.361853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:45.361853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:45.361853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:45.361853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:45.361853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 10:14:45.361853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:45.361853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 10:14:45.361853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:45.361853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 494/997] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:14:45 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:45.364792 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dc60d3-9ab2-4cc6-911a-62c20f507fef None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:45.471100 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dc60d3-9ab2-4cc6-911a-62c20f507fef None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:45.506051 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb426dc-4848-490c-92f4-1adcb60a39a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:45.506679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb426dc-4848-490c-92f4-1adcb60a39a5 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:45.506980 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb426dc-4848-490c-92f4-1adcb60a39a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:45.507258 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb426dc-4848-490c-92f4-1adcb60a39a5 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:45.507933 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb426dc-4848-490c-92f4-1adcb60a39a5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:45.508236 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb426dc-4848-490c-92f4-1adcb60a39a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:45.509608 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb426dc-4848-490c-92f4-1adcb60a39a5 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:45.515303 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 504/998] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 5 10:14:45 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:45.521041 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc2bfe-2a18-4e12-a3ac-4c43ad1bc6cb None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:45.534527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:45.624196 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc2bfe-2a18-4e12-a3ac-4c43ad1bc6cb None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:45.657163 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac00ab7-f6f4-41be-bcc6-6dd5c7b21287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:45.657780 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac00ab7-f6f4-41be-bcc6-6dd5c7b21287 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:45.658037 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac00ab7-f6f4-41be-bcc6-6dd5c7b21287 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:45.658326 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac00ab7-f6f4-41be-bcc6-6dd5c7b21287 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:45.659025 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac00ab7-f6f4-41be-bcc6-6dd5c7b21287 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:45.659335 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac00ab7-f6f4-41be-bcc6-6dd5c7b21287 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:45.660698 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac00ab7-f6f4-41be-bcc6-6dd5c7b21287 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:45.665765 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 495/999] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 5 10:14:45 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:45.672150 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb426dc-4848-490c-92f4-1adcb60a39a5 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:45.701660 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:45.777265 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb426dc-4848-490c-92f4-1adcb60a39a5 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:45.813230 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f325d83d-8ac3-41ce-94af-552396614c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:45.813882 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325d83d-8ac3-41ce-94af-552396614c3c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:45.814160 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325d83d-8ac3-41ce-94af-552396614c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:45.814428 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325d83d-8ac3-41ce-94af-552396614c3c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:45.815240 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f325d83d-8ac3-41ce-94af-552396614c3c None admin] RBAC: Adding query filter params (interface=public, service_id=09a8b1bb31c84774af6e7551e2e946a2, region_id=RegionOne) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:45.815552 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f325d83d-8ac3-41ce-94af-552396614c3c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:45.816967 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f325d83d-8ac3-41ce-94af-552396614c3c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:45.823430 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 505/1000] 10.222.0.40 () {56 vars in 1251 bytes} [Fri Aug 5 10:14:45 2022] GET /identity/v3/endpoints?service_id=09a8b1bb31c84774af6e7551e2e946a2&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:14:45.829061 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac00ab7-f6f4-41be-bcc6-6dd5c7b21287 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:45.911526 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:45.939453 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac00ab7-f6f4-41be-bcc6-6dd5c7b21287 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:45.974289 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cebd8ce-db50-48b8-8631-b3df8e4793c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:45.974940 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cebd8ce-db50-48b8-8631-b3df8e4793c7 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:45.975213 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cebd8ce-db50-48b8-8631-b3df8e4793c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:45.975485 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cebd8ce-db50-48b8-8631-b3df8e4793c7 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:45.976144 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cebd8ce-db50-48b8-8631-b3df8e4793c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:45.977514 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cebd8ce-db50-48b8-8631-b3df8e4793c7 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:45.984544 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 496/1001] 10.222.0.40 () {56 vars in 1087 bytes} [Fri Aug 5 10:14:45 2022] GET /identity/v3/services => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:14:46.712147 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060e8d5-e5f2-4ecd-95d5-2926b1c78489 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:46.712567 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060e8d5-e5f2-4ecd-95d5-2926b1c78489 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:46.712669 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060e8d5-e5f2-4ecd-95d5-2926b1c78489 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:46.714220 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 506/1002] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:46.719811 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24957b9f-1ee3-4e5d-bec4-144fed9e43cb None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:46.720306 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24957b9f-1ee3-4e5d-bec4-144fed9e43cb None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:46.720396 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24957b9f-1ee3-4e5d-bec4-144fed9e43cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:46.739320 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-24957b9f-1ee3-4e5d-bec4-144fed9e43cb None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:46.740521 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-24957b9f-1ee3-4e5d-bec4-144fed9e43cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:46.756989 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff49dea-336e-438d-b06c-e58895f6285c None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:46.757322 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff49dea-336e-438d-b06c-e58895f6285c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:46.757550 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff49dea-336e-438d-b06c-e58895f6285c None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:46.759069 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 507/1003] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 10:14:46.763860 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3b8c2-d88e-4921-8298-c531c0b56f7e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:46.764223 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3b8c2-d88e-4921-8298-c531c0b56f7e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:46.764423 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3b8c2-d88e-4921-8298-c531c0b56f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:46.783833 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6de3b8c2-d88e-4921-8298-c531c0b56f7e None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:46.785010 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6de3b8c2-d88e-4921-8298-c531c0b56f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:46.814713 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 497/1004] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:46 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:14:46.838004 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24957b9f-1ee3-4e5d-bec4-144fed9e43cb None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:46.867185 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 508/1005] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:46 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:14:46.890248 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3b8c2-d88e-4921-8298-c531c0b56f7e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:46.913777 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:46.915280 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:46.951860 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24957b9f-1ee3-4e5d-bec4-144fed9e43cb None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:46.987673 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ac4a13-3912-4858-91d1-d7f438a0af6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:46.988572 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ac4a13-3912-4858-91d1-d7f438a0af6d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:46.989009 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ac4a13-3912-4858-91d1-d7f438a0af6d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:46.989284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ac4a13-3912-4858-91d1-d7f438a0af6d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:46.989914 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ac4a13-3912-4858-91d1-d7f438a0af6d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:46.991319 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ac4a13-3912-4858-91d1-d7f438a0af6d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:46.996966 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3b8c2-d88e-4921-8298-c531c0b56f7e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:47.000521 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-e4ac4a13-3912-4858-91d1-d7f438a0af6d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 10:14:47.000521 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:47.000521 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 10:14:47.000521 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 10:14:47.000521 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 10:14:47.000521 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 10:14:47.000521 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 10:14:47.000521 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:47.000521 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 10:14:47.000521 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:47.000521 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 10:14:47.000521 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:47.000521 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:47.000521 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:47.000521 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:47.000521 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:47.000521 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:47.001704 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:47.001704 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:47.001704 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:47.001704 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:47.001704 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:47.001704 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:47.001704 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 10:14:47.001704 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 10:14:47.001704 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 10:14:47.001704 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 10:14:47.001704 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:47.001704 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:47.001704 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:47.001704 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:47.001704 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:47.001704 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:47.002740 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:47.002740 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:47.002740 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:47.002740 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:47.002740 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:47.002740 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:47.002740 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:47.002740 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:47.002740 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:47.002740 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:47.002740 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 10:14:47.002740 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:47.002740 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 10:14:47.002740 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:47.002740 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 498/1006] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Aug 5 10:14:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:14:47.007594 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ac4a13-3912-4858-91d1-d7f438a0af6d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:47.038199 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9910e47-131f-49f2-a651-4b8ab9dfc24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:47.039107 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9910e47-131f-49f2-a651-4b8ab9dfc24b None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:47.039555 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9910e47-131f-49f2-a651-4b8ab9dfc24b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:47.040043 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9910e47-131f-49f2-a651-4b8ab9dfc24b None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:47.041157 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9910e47-131f-49f2-a651-4b8ab9dfc24b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:47.043065 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9910e47-131f-49f2-a651-4b8ab9dfc24b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:47.060510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 509/1007] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 5 10:14:46 2022] POST /identity/v3/services => generated 242 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:14:47.119726 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ac4a13-3912-4858-91d1-d7f438a0af6d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:47.154451 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:47.168553 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820df7d5-23dc-401f-aa54-5e0cab48d662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:47.169179 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820df7d5-23dc-401f-aa54-5e0cab48d662 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:47.169469 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820df7d5-23dc-401f-aa54-5e0cab48d662 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:47.169736 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820df7d5-23dc-401f-aa54-5e0cab48d662 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:47.170425 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820df7d5-23dc-401f-aa54-5e0cab48d662 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:47.170731 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820df7d5-23dc-401f-aa54-5e0cab48d662 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:47.172195 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820df7d5-23dc-401f-aa54-5e0cab48d662 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:47.178432 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 499/1008] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Aug 5 10:14:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:47.184647 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9910e47-131f-49f2-a651-4b8ab9dfc24b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:47.275454 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:47.291461 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9910e47-131f-49f2-a651-4b8ab9dfc24b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:47.324878 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f995e199-d744-403a-b5d1-df47082ee677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:47.325652 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f995e199-d744-403a-b5d1-df47082ee677 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:47.326027 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f995e199-d744-403a-b5d1-df47082ee677 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:47.326394 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f995e199-d744-403a-b5d1-df47082ee677 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:47.327220 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f995e199-d744-403a-b5d1-df47082ee677 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:47.327619 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f995e199-d744-403a-b5d1-df47082ee677 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:47.329083 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f995e199-d744-403a-b5d1-df47082ee677 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:47.334245 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 510/1009] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Aug 5 10:14:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:47.340028 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820df7d5-23dc-401f-aa54-5e0cab48d662 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:47.442941 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820df7d5-23dc-401f-aa54-5e0cab48d662 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:47.477976 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb817764-1493-4609-a68b-ccf7ee574e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:47.478737 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb817764-1493-4609-a68b-ccf7ee574e96 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:47.479037 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb817764-1493-4609-a68b-ccf7ee574e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:47.479353 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb817764-1493-4609-a68b-ccf7ee574e96 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:47.480033 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb817764-1493-4609-a68b-ccf7ee574e96 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:47.481414 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb817764-1493-4609-a68b-ccf7ee574e96 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:47.499830 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 500/1010] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:14:47 2022] POST /identity/v3/endpoints => generated 325 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:14:47.982145 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb564741-08df-4d5e-aa0d-50f1050bb15b None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:47.982728 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb564741-08df-4d5e-aa0d-50f1050bb15b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:47.982728 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb564741-08df-4d5e-aa0d-50f1050bb15b None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:47.984241 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 511/1011] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:47.989383 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987bf7b9-690b-4788-a618-8f9ae392ad16 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:47.989807 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987bf7b9-690b-4788-a618-8f9ae392ad16 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:47.989892 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987bf7b9-690b-4788-a618-8f9ae392ad16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:48.008754 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-987bf7b9-690b-4788-a618-8f9ae392ad16 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:48.009984 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-987bf7b9-690b-4788-a618-8f9ae392ad16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:48.060535 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:48.089602 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 501/1012] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:47 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:14:48.111909 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb564741-08df-4d5e-aa0d-50f1050bb15b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:48.228773 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb564741-08df-4d5e-aa0d-50f1050bb15b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:48.262735 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d232af-2277-43bc-a4d4-51420240c8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:48.263347 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d232af-2277-43bc-a4d4-51420240c8ab None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:48.263632 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d232af-2277-43bc-a4d4-51420240c8ab None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:48.263940 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d232af-2277-43bc-a4d4-51420240c8ab None admin] PATH_INFO: `/v3/services/compute` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:48.264577 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d232af-2277-43bc-a4d4-51420240c8ab None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:48.265975 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d232af-2277-43bc-a4d4-51420240c8ab None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:48.275120 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-78d232af-2277-43bc-a4d4-51420240c8ab None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 10:14:48.275120 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:48.275120 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 10:14:48.275120 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 10:14:48.275120 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 10:14:48.275120 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 10:14:48.275120 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 10:14:48.275120 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:48.275120 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 10:14:48.275120 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:48.275120 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 10:14:48.275120 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:48.275120 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:48.275120 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:48.275120 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:48.275120 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:48.275120 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:48.276280 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:48.276280 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:48.276280 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:48.276280 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:48.276280 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:48.276280 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:48.276280 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 10:14:48.276280 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 10:14:48.276280 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 10:14:48.276280 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 10:14:48.276280 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:48.276280 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:48.276280 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:48.276280 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:48.276280 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:48.276280 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:48.282226 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:48.282226 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:48.282226 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:48.282226 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:48.282226 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:48.282226 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:48.282226 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:48.282226 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:48.282226 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:48.282226 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:48.282226 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 10:14:48.282226 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:48.282226 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 10:14:48.282226 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:48.282226 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 512/1013] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:14:48 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:48.284838 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987bf7b9-690b-4788-a618-8f9ae392ad16 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:48.386112 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987bf7b9-690b-4788-a618-8f9ae392ad16 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:48.419399 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57b0f01-a21a-4ea6-be55-2be57b822173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:48.420021 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b0f01-a21a-4ea6-be55-2be57b822173 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:48.420297 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b0f01-a21a-4ea6-be55-2be57b822173 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:48.420564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b0f01-a21a-4ea6-be55-2be57b822173 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:48.421235 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57b0f01-a21a-4ea6-be55-2be57b822173 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:48.421534 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57b0f01-a21a-4ea6-be55-2be57b822173 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:48.422911 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57b0f01-a21a-4ea6-be55-2be57b822173 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:48.427902 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 502/1014] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 5 10:14:48 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:48.433731 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d232af-2277-43bc-a4d4-51420240c8ab None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:48.446538 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:48.533531 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d232af-2277-43bc-a4d4-51420240c8ab None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:48.568060 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a72d4-eb6a-418d-944e-5c3cec9eb340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:48.568697 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a72d4-eb6a-418d-944e-5c3cec9eb340 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:48.569063 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a72d4-eb6a-418d-944e-5c3cec9eb340 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:48.569263 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a72d4-eb6a-418d-944e-5c3cec9eb340 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:48.569930 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850a72d4-eb6a-418d-944e-5c3cec9eb340 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:48.570234 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850a72d4-eb6a-418d-944e-5c3cec9eb340 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:48.571642 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850a72d4-eb6a-418d-944e-5c3cec9eb340 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:48.579132 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 513/1015] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 5 10:14:48 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:48.585051 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57b0f01-a21a-4ea6-be55-2be57b822173 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:48.631445 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:48.689839 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57b0f01-a21a-4ea6-be55-2be57b822173 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:48.724136 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebd0215-63c4-473d-a036-4ccd7ad21d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:48.724895 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd0215-63c4-473d-a036-4ccd7ad21d12 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:48.725270 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd0215-63c4-473d-a036-4ccd7ad21d12 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:48.725635 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd0215-63c4-473d-a036-4ccd7ad21d12 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:48.726441 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebd0215-63c4-473d-a036-4ccd7ad21d12 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:48.727937 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebd0215-63c4-473d-a036-4ccd7ad21d12 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:48.748946 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 503/1016] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:14:48 2022] POST /identity/v3/endpoints => generated 334 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:14:49.103506 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f390d-5e82-4dd2-8cdc-229675c2a25c None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:49.104060 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f390d-5e82-4dd2-8cdc-229675c2a25c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:49.104060 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f390d-5e82-4dd2-8cdc-229675c2a25c None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:49.105648 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 514/1017] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:49.110980 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea429694-4677-484d-891e-e111c4442c20 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:49.111418 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea429694-4677-484d-891e-e111c4442c20 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:49.111585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea429694-4677-484d-891e-e111c4442c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:49.130966 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ea429694-4677-484d-891e-e111c4442c20 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:49.132149 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ea429694-4677-484d-891e-e111c4442c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:49.215602 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 504/1018] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:49 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:14:49.238849 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f390d-5e82-4dd2-8cdc-229675c2a25c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:49.318086 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:49.345768 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f390d-5e82-4dd2-8cdc-229675c2a25c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:49.381480 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aa2fe2-fa32-4ba8-a30c-6600d601bdfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:49.382039 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa2fe2-fa32-4ba8-a30c-6600d601bdfb None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:49.382311 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa2fe2-fa32-4ba8-a30c-6600d601bdfb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:49.382573 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa2fe2-fa32-4ba8-a30c-6600d601bdfb None admin] PATH_INFO: `/v3/services/image` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:49.383228 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aa2fe2-fa32-4ba8-a30c-6600d601bdfb None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:49.384727 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aa2fe2-fa32-4ba8-a30c-6600d601bdfb None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:49.393837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-d2aa2fe2-fa32-4ba8-a30c-6600d601bdfb None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 10:14:49.393837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:49.393837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 10:14:49.393837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 10:14:49.393837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 10:14:49.393837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 10:14:49.393837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 10:14:49.393837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:49.393837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 10:14:49.393837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:49.393837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 10:14:49.393837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:49.393837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:49.393837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:49.393837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:49.393837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:49.393837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:49.394862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:49.394862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:49.394862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:49.394862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:49.394862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:49.394862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:49.394862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 10:14:49.394862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 10:14:49.394862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 10:14:49.394862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 10:14:49.394862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:49.394862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:49.394862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:49.394862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:49.394862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:49.394862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:49.395785 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:49.395785 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:49.395785 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:49.395785 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:49.395785 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:49.395785 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:49.395785 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:49.395785 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:49.395785 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:49.395785 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:49.395785 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 10:14:49.395785 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:49.395785 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 10:14:49.395785 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:49.397205 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 515/1019] 10.222.0.40 () {56 vars in 1105 bytes} [Fri Aug 5 10:14:49 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:49.401442 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea429694-4677-484d-891e-e111c4442c20 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:49.426004 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:49.505410 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea429694-4677-484d-891e-e111c4442c20 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:49.534950 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a80ca2-447e-4636-bf80-eef3613d97ba None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:49.535238 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a80ca2-447e-4636-bf80-eef3613d97ba None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:49.535514 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a80ca2-447e-4636-bf80-eef3613d97ba None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:49.537148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 516/1020] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:49.542583 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c35df-f3e2-46cb-91ac-c12b23e3cbcd None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:49.542852 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86860ff-5770-4710-bd97-4dbee2586ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:49.542989 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c35df-f3e2-46cb-91ac-c12b23e3cbcd None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:49.543174 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c35df-f3e2-46cb-91ac-c12b23e3cbcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:49.543529 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86860ff-5770-4710-bd97-4dbee2586ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:49.543808 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86860ff-5770-4710-bd97-4dbee2586ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:49.544074 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86860ff-5770-4710-bd97-4dbee2586ed7 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:49.544759 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86860ff-5770-4710-bd97-4dbee2586ed7 None admin] RBAC: Adding query filter params (name=image) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:49.545061 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86860ff-5770-4710-bd97-4dbee2586ed7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:49.546425 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86860ff-5770-4710-bd97-4dbee2586ed7 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:49.556595 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 505/1021] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:49 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:49.557849 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86860ff-5770-4710-bd97-4dbee2586ed7 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:49.562385 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-737c35df-f3e2-46cb-91ac-c12b23e3cbcd None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:49.563633 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-737c35df-f3e2-46cb-91ac-c12b23e3cbcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:49.639651 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 517/1022] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:49 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 10:14:49.656559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:49.662254 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737c35df-f3e2-46cb-91ac-c12b23e3cbcd None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:49.671794 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86860ff-5770-4710-bd97-4dbee2586ed7 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:49.714473 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e824f-9790-4e30-a1c3-b7034736b3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:49.715097 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e824f-9790-4e30-a1c3-b7034736b3a5 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:49.715374 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e824f-9790-4e30-a1c3-b7034736b3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:49.715640 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e824f-9790-4e30-a1c3-b7034736b3a5 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:49.716311 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8e824f-9790-4e30-a1c3-b7034736b3a5 None admin] RBAC: Adding query filter params (type=image) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:49.716611 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8e824f-9790-4e30-a1c3-b7034736b3a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:49.717972 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8e824f-9790-4e30-a1c3-b7034736b3a5 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:49.723253 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 506/1023] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:49 2022] GET /identity/v3/services?type=image => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:49.729796 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e824f-9790-4e30-a1c3-b7034736b3a5 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:49.733011 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:49.769211 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737c35df-f3e2-46cb-91ac-c12b23e3cbcd None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:49.806222 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17712e-1adb-46b6-ad00-efabbf93f8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:49.807451 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17712e-1adb-46b6-ad00-efabbf93f8d8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:49.808001 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17712e-1adb-46b6-ad00-efabbf93f8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:49.808336 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17712e-1adb-46b6-ad00-efabbf93f8d8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:49.809584 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b17712e-1adb-46b6-ad00-efabbf93f8d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:49.811058 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b17712e-1adb-46b6-ad00-efabbf93f8d8 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:49.812849 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 518/1024] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:14:49.818334 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17712e-1adb-46b6-ad00-efabbf93f8d8 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:49.844014 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e824f-9790-4e30-a1c3-b7034736b3a5 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:49.886303 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f87a02-7c89-49cb-b22f-f3c874cac291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:49.886957 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f87a02-7c89-49cb-b22f-f3c874cac291 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:49.887240 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f87a02-7c89-49cb-b22f-f3c874cac291 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:49.887512 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f87a02-7c89-49cb-b22f-f3c874cac291 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:49.888271 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f87a02-7c89-49cb-b22f-f3c874cac291 None admin] RBAC: Adding query filter params (interface=public, service_id=4ed2bfb15fc84a43b04e7a2f667a1672, region_id=RegionOne) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:49.888578 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f87a02-7c89-49cb-b22f-f3c874cac291 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:49.890001 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f87a02-7c89-49cb-b22f-f3c874cac291 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:49.900319 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 507/1025] 10.222.0.40 () {56 vars in 1251 bytes} [Fri Aug 5 10:14:49 2022] GET /identity/v3/endpoints?service_id=4ed2bfb15fc84a43b04e7a2f667a1672&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:14:49.901800 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f87a02-7c89-49cb-b22f-f3c874cac291 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:49.921918 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:49.927741 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17712e-1adb-46b6-ad00-efabbf93f8d8 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:49.966150 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b613096f-140b-40b5-afd5-49022def76c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:49.966782 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613096f-140b-40b5-afd5-49022def76c6 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:49.967097 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613096f-140b-40b5-afd5-49022def76c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:49.967372 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613096f-140b-40b5-afd5-49022def76c6 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:49.968069 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b613096f-140b-40b5-afd5-49022def76c6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:49.969514 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b613096f-140b-40b5-afd5-49022def76c6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:49.975337 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:50.000767 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 519/1026] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 5 10:14:49 2022] POST /identity/v3/projects => generated 311 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:14:50.019766 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f87a02-7c89-49cb-b22f-f3c874cac291 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:50.057916 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb919313-1bb1-4f62-9632-43624ecfae89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:50.058562 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb919313-1bb1-4f62-9632-43624ecfae89 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:50.058842 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb919313-1bb1-4f62-9632-43624ecfae89 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:50.059147 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb919313-1bb1-4f62-9632-43624ecfae89 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:50.059821 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb919313-1bb1-4f62-9632-43624ecfae89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:50.061236 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb919313-1bb1-4f62-9632-43624ecfae89 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:50.075196 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 508/1027] 10.222.0.40 () {56 vars in 1087 bytes} [Fri Aug 5 10:14:49 2022] GET /identity/v3/services => generated 1957 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:14:50.821108 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151c93e-7c7f-480e-b2c5-2f8b80fcce31 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:50.821662 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151c93e-7c7f-480e-b2c5-2f8b80fcce31 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:50.821662 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151c93e-7c7f-480e-b2c5-2f8b80fcce31 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:50.823289 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 520/1028] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:50.828597 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a941da-8223-40dc-bee5-1a5931f76fc7 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:50.829035 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a941da-8223-40dc-bee5-1a5931f76fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:50.829198 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a941da-8223-40dc-bee5-1a5931f76fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:50.848259 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-03a941da-8223-40dc-bee5-1a5931f76fc7 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:50.849471 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-03a941da-8223-40dc-bee5-1a5931f76fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:50.921806 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 509/1029] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:50 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:14:50.944625 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9151c93e-7c7f-480e-b2c5-2f8b80fcce31 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:51.052758 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9151c93e-7c7f-480e-b2c5-2f8b80fcce31 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:51.086103 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43be9327-ec70-492f-94f8-9f5d3008e847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:51.086670 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43be9327-ec70-492f-94f8-9f5d3008e847 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:51.086965 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43be9327-ec70-492f-94f8-9f5d3008e847 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:51.087240 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43be9327-ec70-492f-94f8-9f5d3008e847 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:51.098025 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43be9327-ec70-492f-94f8-9f5d3008e847 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:14:51.102891 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-43be9327-ec70-492f-94f8-9f5d3008e847 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 10:14:51.102891 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:51.102891 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:51.102891 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:51.102891 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:51.102891 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:51.102891 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:51.102891 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:51.102891 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:51.102891 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:51.102891 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:51.102891 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:51.102891 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:14:51.102891 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:14:51.102891 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:14:51.102891 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:14:51.104059 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:14:51.104059 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:14:51.104059 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:14:51.104059 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:14:51.104059 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:51.104059 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:51.104059 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:51.104059 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:51.104059 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:51.104059 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:51.104059 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:51.104059 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:51.104059 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:51.104059 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:51.104059 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:51.104059 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:51.110280 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:51.110280 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:51.110280 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:51.110280 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:51.110280 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:14:51.110280 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:14:51.110280 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:14:51.110280 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:14:51.110280 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:14:51.110280 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:14:51.110280 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 10:14:51.110280 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:51.110280 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 521/1030] 10.222.0.40 () {56 vars in 1120 bytes} [Fri Aug 5 10:14:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:51.112321 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a941da-8223-40dc-bee5-1a5931f76fc7 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:51.125034 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:51.231811 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a941da-8223-40dc-bee5-1a5931f76fc7 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:51.266336 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116cbce3-8514-4d30-bce2-f6162116a073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:51.266984 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116cbce3-8514-4d30-bce2-f6162116a073 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:51.267270 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116cbce3-8514-4d30-bce2-f6162116a073 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:51.267544 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116cbce3-8514-4d30-bce2-f6162116a073 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:51.268268 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116cbce3-8514-4d30-bce2-f6162116a073 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:51.268573 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116cbce3-8514-4d30-bce2-f6162116a073 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:51.269953 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116cbce3-8514-4d30-bce2-f6162116a073 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:51.278789 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 510/1031] 10.222.0.40 () {56 vars in 1115 bytes} [Fri Aug 5 10:14:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:51.284437 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43be9327-ec70-492f-94f8-9f5d3008e847 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:51.334243 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:51.387720 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43be9327-ec70-492f-94f8-9f5d3008e847 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:51.422477 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9038e677-07f2-4a3d-863a-f8c2b26f9519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:51.423159 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9038e677-07f2-4a3d-863a-f8c2b26f9519 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:51.423441 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9038e677-07f2-4a3d-863a-f8c2b26f9519 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:51.423713 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9038e677-07f2-4a3d-863a-f8c2b26f9519 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:51.435216 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9038e677-07f2-4a3d-863a-f8c2b26f9519 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:51.436675 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9038e677-07f2-4a3d-863a-f8c2b26f9519 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:51.450715 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-9038e677-07f2-4a3d-863a-f8c2b26f9519 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:51.450715 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:51.450715 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:51.450715 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:51.450715 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:51.450715 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:51.450715 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:51.450715 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:51.450715 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:51.450715 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:51.450715 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:51.450715 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:51.450715 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:51.450715 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:51.450715 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:51.451656 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:51.451656 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:51.451656 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:51.451656 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:51.451656 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:51.451656 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:51.451656 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:51.451656 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:51.451656 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:51.451656 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:51.451656 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:51.451656 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:51.451656 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:51.451656 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:51.451656 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:51.451656 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:51.451656 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:51.452562 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:51.452562 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:51.452562 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:51.452562 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:51.453487 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 522/1032] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:51.457642 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116cbce3-8514-4d30-bce2-f6162116a073 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:51.533838 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:51.565262 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116cbce3-8514-4d30-bce2-f6162116a073 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:51.600604 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4ea0b-d6a0-4c99-a61b-3f70c3d334f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:51.601225 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4ea0b-d6a0-4c99-a61b-3f70c3d334f6 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:51.601500 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4ea0b-d6a0-4c99-a61b-3f70c3d334f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:51.601770 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4ea0b-d6a0-4c99-a61b-3f70c3d334f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:51.602458 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e4ea0b-d6a0-4c99-a61b-3f70c3d334f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:51.602755 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e4ea0b-d6a0-4c99-a61b-3f70c3d334f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:51.604232 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e4ea0b-d6a0-4c99-a61b-3f70c3d334f6 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:51.621089 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 511/1033] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:51.622404 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9038e677-07f2-4a3d-863a-f8c2b26f9519 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:51.722753 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:51.726571 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9038e677-07f2-4a3d-863a-f8c2b26f9519 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:51.762205 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278ce23-7c06-40c5-96a1-fc153f667abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:51.762846 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278ce23-7c06-40c5-96a1-fc153f667abf None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:51.763145 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278ce23-7c06-40c5-96a1-fc153f667abf None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:51.763415 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278ce23-7c06-40c5-96a1-fc153f667abf None admin] PATH_INFO: `/v3/users/nova` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:51.771228 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d278ce23-7c06-40c5-96a1-fc153f667abf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:51.772673 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d278ce23-7c06-40c5-96a1-fc153f667abf None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:51.784150 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-d278ce23-7c06-40c5-96a1-fc153f667abf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 10:14:51.784150 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:51.784150 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:51.784150 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:51.784150 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:51.784150 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:51.784150 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:51.784150 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:51.784150 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:51.784150 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:51.784150 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:51.784150 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:51.784150 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 10:14:51.784150 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 10:14:51.784150 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 10:14:51.784150 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 10:14:51.785141 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:51.785141 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:51.785141 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 10:14:51.785141 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:51.785141 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 10:14:51.785141 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:51.785141 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:51.785141 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:51.785141 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:51.785141 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:51.785141 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:51.785141 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:51.785141 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:51.785141 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:51.785141 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:51.785141 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:51.785141 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:51.785141 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:51.791331 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:51.791331 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:51.791331 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 10:14:51.791331 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 10:14:51.791331 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 10:14:51.791331 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 10:14:51.791331 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 10:14:51.791331 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 10:14:51.791331 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 10:14:51.791331 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:51.791331 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 523/1034] 10.222.0.40 () {56 vars in 1093 bytes} [Fri Aug 5 10:14:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:14:51.792300 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4ea0b-d6a0-4c99-a61b-3f70c3d334f6 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:51.895996 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4ea0b-d6a0-4c99-a61b-3f70c3d334f6 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:51.926341 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:51.932477 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8a20d9-caf2-427a-b07c-205a4df6e7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:51.933102 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a20d9-caf2-427a-b07c-205a4df6e7b8 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:51.933380 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a20d9-caf2-427a-b07c-205a4df6e7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:51.933649 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a20d9-caf2-427a-b07c-205a4df6e7b8 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:51.934443 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8a20d9-caf2-427a-b07c-205a4df6e7b8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:51.934744 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8a20d9-caf2-427a-b07c-205a4df6e7b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:51.936205 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8a20d9-caf2-427a-b07c-205a4df6e7b8 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:51.955252 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 512/1035] 10.222.0.40 () {56 vars in 1133 bytes} [Fri Aug 5 10:14:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:14:51.960916 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278ce23-7c06-40c5-96a1-fc153f667abf None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:52.059190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e3eb3-2b66-4827-b996-340d18e6908e None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:52.059400 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e3eb3-2b66-4827-b996-340d18e6908e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:52.059600 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e3eb3-2b66-4827-b996-340d18e6908e None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:52.061155 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 513/1036] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 10:14:52.065827 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe663b-d32c-456d-b54a-1bbfa7d0b7db None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:52.066109 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe663b-d32c-456d-b54a-1bbfa7d0b7db None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:52.066357 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe663b-d32c-456d-b54a-1bbfa7d0b7db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:52.066809 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278ce23-7c06-40c5-96a1-fc153f667abf None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:52.085304 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-85fe663b-d32c-456d-b54a-1bbfa7d0b7db None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:52.086466 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-85fe663b-d32c-456d-b54a-1bbfa7d0b7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:52.101623 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db5ed0e-59b8-42f3-a333-24ad0c73f288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:52.102251 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db5ed0e-59b8-42f3-a333-24ad0c73f288 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:52.102526 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db5ed0e-59b8-42f3-a333-24ad0c73f288 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:52.102791 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db5ed0e-59b8-42f3-a333-24ad0c73f288 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:52.131412 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:52.157072 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 514/1037] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:52 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:14:52.158559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8b43f-ca1a-4958-bb54-227bba02de36 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:52.159354 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8b43f-ca1a-4958-bb54-227bba02de36 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:52.159354 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8b43f-ca1a-4958-bb54-227bba02de36 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:52.160563 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 515/1038] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:52.166145 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc862348-1142-4b4e-9bbf-f6be2ddcb8c3 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:52.167181 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc862348-1142-4b4e-9bbf-f6be2ddcb8c3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:52.167474 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc862348-1142-4b4e-9bbf-f6be2ddcb8c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:52.176983 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db5ed0e-59b8-42f3-a333-24ad0c73f288 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:52.178609 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db5ed0e-59b8-42f3-a333-24ad0c73f288 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:52.186098 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-cc862348-1142-4b4e-9bbf-f6be2ddcb8c3 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:52.187300 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-cc862348-1142-4b4e-9bbf-f6be2ddcb8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:52.251477 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 516/1039] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:52 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 10:14:52.252295 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc862348-1142-4b4e-9bbf-f6be2ddcb8c3 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:52.281517 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 524/1040] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:14:51 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 10:14:52.282359 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db5ed0e-59b8-42f3-a333-24ad0c73f288 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:52.311258 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:52.342361 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:52.361798 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc862348-1142-4b4e-9bbf-f6be2ddcb8c3 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:52.383139 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db5ed0e-59b8-42f3-a333-24ad0c73f288 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:52.396225 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b817e190-69a1-4ad7-a98e-c72b93fed1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:52.396868 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b817e190-69a1-4ad7-a98e-c72b93fed1ce None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:52.397140 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b817e190-69a1-4ad7-a98e-c72b93fed1ce None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:52.397406 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b817e190-69a1-4ad7-a98e-c72b93fed1ce None admin] PATH_INFO: `/v3/services/image` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:52.398030 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b817e190-69a1-4ad7-a98e-c72b93fed1ce None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:52.399434 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b817e190-69a1-4ad7-a98e-c72b93fed1ce None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:52.408291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-b817e190-69a1-4ad7-a98e-c72b93fed1ce None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 10:14:52.408291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:52.408291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 10:14:52.408291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 10:14:52.408291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 10:14:52.408291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 10:14:52.408291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 10:14:52.408291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:52.408291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 10:14:52.408291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:52.408291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 10:14:52.408291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:52.408291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:52.408291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:52.408291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:52.408291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:52.408291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:52.409445 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:52.409445 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:52.409445 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:52.409445 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:52.409445 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:52.409445 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:52.409445 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 10:14:52.409445 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 10:14:52.409445 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 10:14:52.409445 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 10:14:52.409445 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:52.409445 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:52.409445 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:52.409445 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:52.409445 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:52.409445 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:52.410288 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:52.410288 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:52.410288 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:52.410288 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:52.410288 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:52.410288 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:52.410288 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:52.410288 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:52.410288 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:52.410288 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:52.410288 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 10:14:52.410288 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:14:52.410288 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 10:14:52.410288 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:52.410288 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 517/1041] 10.222.0.40 () {56 vars in 1105 bytes} [Fri Aug 5 10:14:52 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:52.411110 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b817e190-69a1-4ad7-a98e-c72b93fed1ce None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:52.421824 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d829cee-3e82-4f5d-9481-b3ec3806706c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:52.422482 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d829cee-3e82-4f5d-9481-b3ec3806706c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:52.422759 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d829cee-3e82-4f5d-9481-b3ec3806706c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:52.423067 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d829cee-3e82-4f5d-9481-b3ec3806706c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:52.424324 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d829cee-3e82-4f5d-9481-b3ec3806706c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:52.425832 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d829cee-3e82-4f5d-9481-b3ec3806706c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:52.432371 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 525/1042] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:52.432371 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d829cee-3e82-4f5d-9481-b3ec3806706c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:52.510095 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b817e190-69a1-4ad7-a98e-c72b93fed1ce None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:52.523637 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:52.528163 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d829cee-3e82-4f5d-9481-b3ec3806706c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:52.544776 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a4ca84-79b1-4944-afdf-4e2b1cc29023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:52.545424 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4ca84-79b1-4944-afdf-4e2b1cc29023 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:52.545707 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4ca84-79b1-4944-afdf-4e2b1cc29023 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:52.545980 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4ca84-79b1-4944-afdf-4e2b1cc29023 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:52.556096 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:52.556784 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a4ca84-79b1-4944-afdf-4e2b1cc29023 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:52.557969 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a4ca84-79b1-4944-afdf-4e2b1cc29023 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:52.564900 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659ac3f5-de09-4b65-bad4-f5ee2b998771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:52.565463 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659ac3f5-de09-4b65-bad4-f5ee2b998771 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:52.565740 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659ac3f5-de09-4b65-bad4-f5ee2b998771 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:52.566013 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659ac3f5-de09-4b65-bad4-f5ee2b998771 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:52.566685 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659ac3f5-de09-4b65-bad4-f5ee2b998771 None admin] RBAC: Adding query filter params (name=image) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:52.567050 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659ac3f5-de09-4b65-bad4-f5ee2b998771 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:52.568402 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659ac3f5-de09-4b65-bad4-f5ee2b998771 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:52.572488 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-a5a4ca84-79b1-4944-afdf-4e2b1cc29023 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:52.572488 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:52.572488 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:52.572488 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:52.572488 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:52.572488 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:52.572488 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:52.572488 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:52.572488 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:52.572488 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:52.572488 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:52.572488 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:52.572488 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:52.572488 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:52.572488 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:52.573559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:52.573559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:52.573559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:52.573559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:52.573559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:52.573559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:52.573559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:52.573559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:52.573559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:52.573559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:52.573559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:52.573559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:52.573559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:52.573559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:52.573559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:52.573559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:52.573559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:52.574463 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:52.574463 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:52.574463 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:52.574463 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:52.574463 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 518/1043] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:52.574781 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 526/1044] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:52 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:52.574852 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a4ca84-79b1-4944-afdf-4e2b1cc29023 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:52.583740 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659ac3f5-de09-4b65-bad4-f5ee2b998771 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:52.682211 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659ac3f5-de09-4b65-bad4-f5ee2b998771 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:52.686649 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a4ca84-79b1-4944-afdf-4e2b1cc29023 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:52.717397 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242be604-4324-4448-a370-5e7bdd0f298c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:52.718016 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242be604-4324-4448-a370-5e7bdd0f298c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:52.718290 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242be604-4324-4448-a370-5e7bdd0f298c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:52.718556 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242be604-4324-4448-a370-5e7bdd0f298c None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:52.719255 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242be604-4324-4448-a370-5e7bdd0f298c None admin] RBAC: Adding query filter params (type=image) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:52.719551 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242be604-4324-4448-a370-5e7bdd0f298c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:52.720913 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242be604-4324-4448-a370-5e7bdd0f298c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:52.722839 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de0aa6f-e351-4712-844c-534ee8fc231e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:52.723502 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0aa6f-e351-4712-844c-534ee8fc231e None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:52.723786 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0aa6f-e351-4712-844c-534ee8fc231e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:52.724063 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0aa6f-e351-4712-844c-534ee8fc231e None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:52.724746 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de0aa6f-e351-4712-844c-534ee8fc231e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:52.725702 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 527/1045] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:52 2022] GET /identity/v3/services?type=image => generated 350 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:52.726459 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242be604-4324-4448-a370-5e7bdd0f298c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:52.727690 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de0aa6f-e351-4712-844c-534ee8fc231e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:52.750858 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 519/1046] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 5 10:14:52 2022] POST /identity/v3/users => generated 285 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:14:52.751672 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de0aa6f-e351-4712-844c-534ee8fc231e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:52.790153 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:52.798605 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:52.851922 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242be604-4324-4448-a370-5e7bdd0f298c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:52.881336 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de0aa6f-e351-4712-844c-534ee8fc231e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:52.893124 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db094783-801d-4a28-a6a8-0436d3863864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:52.893770 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db094783-801d-4a28-a6a8-0436d3863864 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:52.894050 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db094783-801d-4a28-a6a8-0436d3863864 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:52.894323 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db094783-801d-4a28-a6a8-0436d3863864 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:52.895054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db094783-801d-4a28-a6a8-0436d3863864 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:52.895358 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db094783-801d-4a28-a6a8-0436d3863864 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:52.896770 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db094783-801d-4a28-a6a8-0436d3863864 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:52.909940 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 528/1047] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:52.915628 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db094783-801d-4a28-a6a8-0436d3863864 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:52.926384 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e5f3f-c0b3-40a0-ab68-50c5ba785630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:52.927477 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e5f3f-c0b3-40a0-ab68-50c5ba785630 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:52.927760 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e5f3f-c0b3-40a0-ab68-50c5ba785630 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:52.928034 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e5f3f-c0b3-40a0-ab68-50c5ba785630 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:52.928752 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7e5f3f-c0b3-40a0-ab68-50c5ba785630 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:52.930134 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7e5f3f-c0b3-40a0-ab68-50c5ba785630 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:52.955776 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 520/1048] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:14:52 2022] POST /identity/v3/endpoints => generated 327 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:14:53.038708 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db094783-801d-4a28-a6a8-0436d3863864 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:53.058040 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:53.085434 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4579478-f927-4538-a315-d95df2592ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:53.086104 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4579478-f927-4538-a315-d95df2592ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:53.086382 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4579478-f927-4538-a315-d95df2592ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:53.086655 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4579478-f927-4538-a315-d95df2592ff4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:53.094673 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4579478-f927-4538-a315-d95df2592ff4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:53.096196 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4579478-f927-4538-a315-d95df2592ff4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:53.108894 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-e4579478-f927-4538-a315-d95df2592ff4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:53.108894 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:53.108894 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:53.108894 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:53.108894 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:53.108894 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:53.108894 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:53.108894 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:53.108894 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:53.108894 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:53.108894 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:53.108894 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:53.108894 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 10:14:53.108894 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 10:14:53.108894 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 10:14:53.109856 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 10:14:53.109856 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:53.109856 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:53.109856 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:53.109856 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:53.109856 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:53.109856 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:53.109856 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:53.109856 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:53.109856 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:53.109856 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:53.109856 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:53.109856 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:53.109856 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:53.109856 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:53.109856 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:53.109856 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:53.110978 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 10:14:53.110978 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 10:14:53.110978 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 10:14:53.110978 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 10:14:53.110978 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 10:14:53.110978 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 10:14:53.110978 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:53.110978 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:53.110978 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 529/1049] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:14:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:53.117418 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e5f3f-c0b3-40a0-ab68-50c5ba785630 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:53.207363 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:53.232359 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e5f3f-c0b3-40a0-ab68-50c5ba785630 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:53.268420 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b284d95-85ee-457c-95f5-eb203d3b31c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:53.269051 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b284d95-85ee-457c-95f5-eb203d3b31c7 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:53.269333 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b284d95-85ee-457c-95f5-eb203d3b31c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:53.269608 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b284d95-85ee-457c-95f5-eb203d3b31c7 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:53.270367 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b284d95-85ee-457c-95f5-eb203d3b31c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:53.270677 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b284d95-85ee-457c-95f5-eb203d3b31c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:53.272178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b284d95-85ee-457c-95f5-eb203d3b31c7 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:53.285012 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 521/1050] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 10:14:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:53.290974 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4579478-f927-4538-a315-d95df2592ff4 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:53.394196 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4579478-f927-4538-a315-d95df2592ff4 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:53.427712 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c60cfc-a9e3-4467-ab79-55986c83d704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:53.428334 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c60cfc-a9e3-4467-ab79-55986c83d704 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:53.428607 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c60cfc-a9e3-4467-ab79-55986c83d704 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:53.428872 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c60cfc-a9e3-4467-ab79-55986c83d704 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:53.429652 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c60cfc-a9e3-4467-ab79-55986c83d704 None admin] RBAC: Adding query filter params (role.id=f10badb1efd340eda8d6358dc6bd9c28, scope.project.id=36b8b505012d4d0bac54845b6799e743, user.id=b938f1f41db94b0eafe95bc8eeaae175) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:53.429954 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c60cfc-a9e3-4467-ab79-55986c83d704 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:53.431438 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c60cfc-a9e3-4467-ab79-55986c83d704 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:53.437312 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 530/1051] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:14:53 2022] GET /identity/v3/role_assignments?user.id=b938f1f41db94b0eafe95bc8eeaae175&scope.project.id=36b8b505012d4d0bac54845b6799e743&role.id=f10badb1efd340eda8d6358dc6bd9c28 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:14:54.907411 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b39b1d-3daf-4aec-8ace-690c3f166d9e None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:54.907985 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b39b1d-3daf-4aec-8ace-690c3f166d9e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:54.907985 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b39b1d-3daf-4aec-8ace-690c3f166d9e None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:54.909449 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 522/1052] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:54.917891 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711256c6-b345-4d33-aa83-942e06a9c78b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:54.918349 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711256c6-b345-4d33-aa83-942e06a9c78b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:54.918438 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711256c6-b345-4d33-aa83-942e06a9c78b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:54.937574 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-711256c6-b345-4d33-aa83-942e06a9c78b None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:54.938770 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-711256c6-b345-4d33-aa83-942e06a9c78b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:54.971167 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:54.990113 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446389f-a597-4bb6-9f50-999a76516e61 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:54.990345 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446389f-a597-4bb6-9f50-999a76516e61 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:54.990629 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446389f-a597-4bb6-9f50-999a76516e61 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:54.992022 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 523/1053] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:54.996983 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a4129-23f7-472c-9213-22750d27c443 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:54.997259 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a4129-23f7-472c-9213-22750d27c443 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:54.997528 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a4129-23f7-472c-9213-22750d27c443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:55.013525 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 531/1054] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:14:55.016573 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-006a4129-23f7-472c-9213-22750d27c443 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:55.017747 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-006a4129-23f7-472c-9213-22750d27c443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:55.036417 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711256c6-b345-4d33-aa83-942e06a9c78b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:55.091078 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 524/1055] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 10:14:55.110682 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a4129-23f7-472c-9213-22750d27c443 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:55.145159 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711256c6-b345-4d33-aa83-942e06a9c78b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:55.183554 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175243a5-5ab7-4e43-8d24-a044c5f679c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:55.184216 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175243a5-5ab7-4e43-8d24-a044c5f679c7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:55.184494 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175243a5-5ab7-4e43-8d24-a044c5f679c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:55.184761 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175243a5-5ab7-4e43-8d24-a044c5f679c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:55.186772 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:55.195290 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175243a5-5ab7-4e43-8d24-a044c5f679c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:14:55.200781 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-175243a5-5ab7-4e43-8d24-a044c5f679c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:14:55.200781 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:55.200781 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:55.200781 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:55.200781 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:55.200781 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:55.200781 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:55.200781 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:55.200781 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:55.200781 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:55.200781 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:55.200781 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:55.200781 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:14:55.200781 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:14:55.200781 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:14:55.200781 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:14:55.201901 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:14:55.201901 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:14:55.201901 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:14:55.201901 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:14:55.201901 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:55.201901 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:55.201901 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:55.201901 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:55.201901 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:55.201901 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:55.201901 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:55.201901 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:55.201901 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:55.201901 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:55.201901 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:55.201901 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:55.202800 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:55.202800 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:55.202800 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:55.202800 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:55.202800 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:14:55.202800 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:14:55.202800 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:14:55.202800 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:14:55.202800 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:14:55.202800 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:14:55.202800 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:14:55.202800 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:55.202800 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 532/1056] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 10:14:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:14:55.208071 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175243a5-5ab7-4e43-8d24-a044c5f679c7 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:55.226678 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a4129-23f7-472c-9213-22750d27c443 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:55.236763 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:55.262973 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1028722c-c467-45cf-a736-fd9aac649447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:55.263614 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028722c-c467-45cf-a736-fd9aac649447 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:55.263887 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028722c-c467-45cf-a736-fd9aac649447 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:55.264213 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028722c-c467-45cf-a736-fd9aac649447 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:55.317567 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175243a5-5ab7-4e43-8d24-a044c5f679c7 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:55.334801 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1028722c-c467-45cf-a736-fd9aac649447 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:55.336299 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1028722c-c467-45cf-a736-fd9aac649447 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:55.363820 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952673b5-cda8-4525-9125-2f09c7cb72fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:55.364456 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952673b5-cda8-4525-9125-2f09c7cb72fb None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:55.364738 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952673b5-cda8-4525-9125-2f09c7cb72fb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:55.365448 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952673b5-cda8-4525-9125-2f09c7cb72fb None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:55.366218 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952673b5-cda8-4525-9125-2f09c7cb72fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:55.366673 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952673b5-cda8-4525-9125-2f09c7cb72fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:55.368182 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952673b5-cda8-4525-9125-2f09c7cb72fb None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:55.378222 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 533/1057] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:14:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:55.385038 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6319be-77c7-4b81-ba32-3a59f9e3a14c None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:55.385347 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6319be-77c7-4b81-ba32-3a59f9e3a14c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:55.385627 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6319be-77c7-4b81-ba32-3a59f9e3a14c None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:55.386922 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 534/1058] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 10:14:55.387678 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6319be-77c7-4b81-ba32-3a59f9e3a14c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:55.415506 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:55.454952 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 525/1059] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:14:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 05 10:14:55.454952 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1783c4c9-1f65-4a45-a82c-b921a407b669 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:55.454952 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1783c4c9-1f65-4a45-a82c-b921a407b669 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:55.454952 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1783c4c9-1f65-4a45-a82c-b921a407b669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:55.472089 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-1783c4c9-1f65-4a45-a82c-b921a407b669 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:55.473268 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1783c4c9-1f65-4a45-a82c-b921a407b669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:55.481901 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:55.496505 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6319be-77c7-4b81-ba32-3a59f9e3a14c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:55.531254 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ccc473-af59-4687-af3a-7ae51c212d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:55.531917 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ccc473-af59-4687-af3a-7ae51c212d25 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:55.532192 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ccc473-af59-4687-af3a-7ae51c212d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:55.532457 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ccc473-af59-4687-af3a-7ae51c212d25 None admin] PATH_INFO: `/v3/users/391d6d36ccfe472a98b07f33e57a897d` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:55.541178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 526/1060] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:14:55.542436 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1783c4c9-1f65-4a45-a82c-b921a407b669 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:55.546601 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ccc473-af59-4687-af3a-7ae51c212d25 None admin] RBAC: Authorizing `identity:get_user(user_id=391d6d36ccfe472a98b07f33e57a897d)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:55.548133 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ccc473-af59-4687-af3a-7ae51c212d25 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:55.554636 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 535/1061] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:14:55 2022] GET /identity/v3/users/391d6d36ccfe472a98b07f33e57a897d => generated 285 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:55.555929 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ccc473-af59-4687-af3a-7ae51c212d25 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:55.642254 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:55.646231 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1783c4c9-1f65-4a45-a82c-b921a407b669 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:55.654922 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ccc473-af59-4687-af3a-7ae51c212d25 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:55.680836 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860dd89f-9a4e-4638-b1db-6bb4299aec24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:55.681508 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860dd89f-9a4e-4638-b1db-6bb4299aec24 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:55.681783 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860dd89f-9a4e-4638-b1db-6bb4299aec24 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:55.682047 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860dd89f-9a4e-4638-b1db-6bb4299aec24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:55.690429 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b07924d-662b-4fef-9a80-014819dcd127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:55.691083 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b07924d-662b-4fef-9a80-014819dcd127 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:55.691366 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b07924d-662b-4fef-9a80-014819dcd127 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:55.691640 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b07924d-662b-4fef-9a80-014819dcd127 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:55.692097 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860dd89f-9a4e-4638-b1db-6bb4299aec24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:55.693564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860dd89f-9a4e-4638-b1db-6bb4299aec24 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:55.706859 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:55.707924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-860dd89f-9a4e-4638-b1db-6bb4299aec24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:55.707924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:55.707924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:55.707924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:55.707924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:55.707924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:55.707924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:55.707924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:55.707924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:55.707924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:55.707924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:55.707924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:55.707924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:55.707924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:55.707924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:55.708849 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:55.708849 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:55.708849 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:55.708849 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:55.708849 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:55.708849 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:55.708849 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:55.708849 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:55.708849 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:55.708849 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:55.708849 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:55.708849 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:55.708849 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:55.708849 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:55.708849 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:55.708849 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:55.708849 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:55.709731 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:55.709731 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:55.709731 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:55.709731 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:55.709731 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 527/1062] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:55.710636 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860dd89f-9a4e-4638-b1db-6bb4299aec24 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:55.766464 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b07924d-662b-4fef-9a80-014819dcd127 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:55.767969 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b07924d-662b-4fef-9a80-014819dcd127 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:55.811791 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860dd89f-9a4e-4638-b1db-6bb4299aec24 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:55.843629 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8b5beb-acbc-42c4-9bd2-f45730f32815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:55.844261 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b5beb-acbc-42c4-9bd2-f45730f32815 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:55.844540 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b5beb-acbc-42c4-9bd2-f45730f32815 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:55.844808 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b5beb-acbc-42c4-9bd2-f45730f32815 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:55.855585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8b5beb-acbc-42c4-9bd2-f45730f32815 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:14:55.862089 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-2a8b5beb-acbc-42c4-9bd2-f45730f32815 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 10:14:55.862089 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:55.862089 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:55.862089 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:55.862089 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:55.862089 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:55.862089 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:55.862089 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:55.862089 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:55.862089 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:55.862089 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:55.862089 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:55.862089 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:14:55.862089 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:14:55.862089 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:14:55.862089 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:14:55.863139 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:14:55.863139 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:14:55.863139 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:14:55.863139 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:14:55.863139 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:55.863139 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:55.863139 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:55.863139 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:55.863139 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:55.863139 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:55.863139 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:55.863139 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:55.863139 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:55.863139 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:55.863139 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:55.863139 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:55.864382 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:55.864382 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:55.864382 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:55.864382 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:55.864382 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:14:55.864382 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:14:55.864382 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:14:55.864382 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:14:55.864382 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:14:55.864382 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:14:55.864382 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 10:14:55.864382 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:55.865266 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 536/1063] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:14:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 05 10:14:55.866094 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b07924d-662b-4fef-9a80-014819dcd127 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:55.867186 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 528/1064] 10.222.0.40 () {56 vars in 1120 bytes} [Fri Aug 5 10:14:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:55.871306 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8b5beb-acbc-42c4-9bd2-f45730f32815 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:55.883673 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:55.915142 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:55.969202 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b07924d-662b-4fef-9a80-014819dcd127 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:55.971622 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8b5beb-acbc-42c4-9bd2-f45730f32815 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:56.002412 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a2121-7880-4890-8455-b63a91eb2651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:56.003067 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a2121-7880-4890-8455-b63a91eb2651 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:56.003347 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a2121-7880-4890-8455-b63a91eb2651 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:56.003612 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a2121-7880-4890-8455-b63a91eb2651 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:56.004292 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8a2121-7880-4890-8455-b63a91eb2651 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:56.004589 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8a2121-7880-4890-8455-b63a91eb2651 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:56.004742 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a83c6-cfa1-4242-9cea-6ea460619398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:56.005382 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a83c6-cfa1-4242-9cea-6ea460619398 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:56.005656 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a83c6-cfa1-4242-9cea-6ea460619398 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:56.005925 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a83c6-cfa1-4242-9cea-6ea460619398 None admin] PATH_INFO: `/v3/projects/de6e3edce72b44ac83884bb959e47f2b` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:56.006016 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8a2121-7880-4890-8455-b63a91eb2651 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:56.007105 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24a83c6-cfa1-4242-9cea-6ea460619398 None admin] RBAC: Authorizing `identity:get_project(project_id=de6e3edce72b44ac83884bb959e47f2b)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:56.008547 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24a83c6-cfa1-4242-9cea-6ea460619398 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:56.010715 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 529/1065] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:14:55 2022] GET /identity/v3/projects/de6e3edce72b44ac83884bb959e47f2b => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:14:56.011497 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a83c6-cfa1-4242-9cea-6ea460619398 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:56.028023 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 537/1066] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:56.028023 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a2121-7880-4890-8455-b63a91eb2651 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:56.105164 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:56.129288 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a83c6-cfa1-4242-9cea-6ea460619398 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:56.139569 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a2121-7880-4890-8455-b63a91eb2651 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:56.148707 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:56.173131 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d1c25-dc10-4b48-b2b0-ad2e3e622d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:56.173131 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d1c25-dc10-4b48-b2b0-ad2e3e622d62 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:56.173131 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d1c25-dc10-4b48-b2b0-ad2e3e622d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:56.173522 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d1c25-dc10-4b48-b2b0-ad2e3e622d62 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:56.174008 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3d1c25-dc10-4b48-b2b0-ad2e3e622d62 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:56.174301 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3d1c25-dc10-4b48-b2b0-ad2e3e622d62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:56.176233 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3d1c25-dc10-4b48-b2b0-ad2e3e622d62 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:56.185227 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 530/1067] 10.222.0.40 () {56 vars in 1115 bytes} [Fri Aug 5 10:14:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:56.188678 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ebcf1f-a389-4f0b-a604-7de60e6b694c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:56.189299 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ebcf1f-a389-4f0b-a604-7de60e6b694c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:56.189574 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ebcf1f-a389-4f0b-a604-7de60e6b694c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:56.189841 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ebcf1f-a389-4f0b-a604-7de60e6b694c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:56.190628 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ebcf1f-a389-4f0b-a604-7de60e6b694c None admin] RBAC: Adding query filter params (role.id=daef89adcaeb48a382a0bcb03340ccbb, scope.project.id=de6e3edce72b44ac83884bb959e47f2b, user.id=391d6d36ccfe472a98b07f33e57a897d) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:56.191746 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ebcf1f-a389-4f0b-a604-7de60e6b694c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:56.191855 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d1c25-dc10-4b48-b2b0-ad2e3e622d62 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:56.192475 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ebcf1f-a389-4f0b-a604-7de60e6b694c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:56.199519 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 538/1068] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:14:56 2022] GET /identity/v3/role_assignments?user.id=391d6d36ccfe472a98b07f33e57a897d&scope.project.id=de6e3edce72b44ac83884bb959e47f2b&role.id=daef89adcaeb48a382a0bcb03340ccbb => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:56.307780 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d1c25-dc10-4b48-b2b0-ad2e3e622d62 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:56.348029 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cee81fa-ea8c-4167-a83e-73b47f155565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:56.348686 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cee81fa-ea8c-4167-a83e-73b47f155565 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:56.348962 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cee81fa-ea8c-4167-a83e-73b47f155565 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:56.349228 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cee81fa-ea8c-4167-a83e-73b47f155565 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:56.358511 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:56.360890 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cee81fa-ea8c-4167-a83e-73b47f155565 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:56.362823 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cee81fa-ea8c-4167-a83e-73b47f155565 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:56.377290 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-0cee81fa-ea8c-4167-a83e-73b47f155565 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:56.377290 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:56.377290 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:56.377290 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:56.377290 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:56.377290 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:56.377290 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:56.377290 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:56.377290 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:56.377290 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:56.377290 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:56.377290 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:56.377290 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:56.377290 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:56.377290 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:56.378358 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:56.378358 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:56.378358 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:56.378358 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:56.378358 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:56.378358 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:56.378358 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:56.378358 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:56.378358 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:56.378358 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:56.378358 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:56.378358 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:56.378358 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:56.378358 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:56.378358 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:56.378358 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:56.378358 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:56.379666 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:56.379666 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:56.379666 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:56.379666 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:56.379666 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 531/1069] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:14:56.384342 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ebcf1f-a389-4f0b-a604-7de60e6b694c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:56.484988 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ebcf1f-a389-4f0b-a604-7de60e6b694c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:56.520030 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d03b006-0ee2-4181-9c8b-305e7f0b9a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:56.520651 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d03b006-0ee2-4181-9c8b-305e7f0b9a2e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:56.520932 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d03b006-0ee2-4181-9c8b-305e7f0b9a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:56.521205 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d03b006-0ee2-4181-9c8b-305e7f0b9a2e None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:56.521898 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d03b006-0ee2-4181-9c8b-305e7f0b9a2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:56.522199 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d03b006-0ee2-4181-9c8b-305e7f0b9a2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:56.523625 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d03b006-0ee2-4181-9c8b-305e7f0b9a2e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:56.535883 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 539/1070] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:56.542059 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cee81fa-ea8c-4167-a83e-73b47f155565 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:56.645692 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cee81fa-ea8c-4167-a83e-73b47f155565 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:56.682066 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c73951-f3eb-4360-ae89-fa41aa2b3622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:56.682705 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c73951-f3eb-4360-ae89-fa41aa2b3622 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:56.683017 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c73951-f3eb-4360-ae89-fa41aa2b3622 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:56.683292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c73951-f3eb-4360-ae89-fa41aa2b3622 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:56.691121 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c73951-f3eb-4360-ae89-fa41aa2b3622 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:56.692562 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c73951-f3eb-4360-ae89-fa41aa2b3622 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:56.704083 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-56c73951-f3eb-4360-ae89-fa41aa2b3622 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 10:14:56.704083 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:56.704083 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:56.704083 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:56.704083 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:56.704083 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:56.704083 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:56.704083 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:56.704083 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:56.704083 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:56.704083 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:56.704083 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:56.704083 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 10:14:56.704083 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 10:14:56.704083 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 10:14:56.704083 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 10:14:56.705163 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:56.705163 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:56.705163 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 10:14:56.705163 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:56.705163 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 10:14:56.705163 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:14:56.705163 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:56.705163 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:56.705163 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:56.705163 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:56.705163 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:56.705163 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:56.705163 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:56.705163 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:56.705163 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:56.705163 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:56.705163 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:56.705163 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:56.706650 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:56.706650 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:56.706650 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 10:14:56.706650 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 10:14:56.706650 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 10:14:56.706650 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 10:14:56.706650 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 10:14:56.706650 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 10:14:56.706650 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 10:14:56.706650 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:56.711343 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 532/1071] 10.222.0.40 () {56 vars in 1093 bytes} [Fri Aug 5 10:14:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:56.713651 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d03b006-0ee2-4181-9c8b-305e7f0b9a2e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:56.745247 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:56.814684 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d03b006-0ee2-4181-9c8b-305e7f0b9a2e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:56.849751 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a01f0f-f8e5-448b-a86e-07e4368d8dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:56.850373 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a01f0f-f8e5-448b-a86e-07e4368d8dc3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:56.850654 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a01f0f-f8e5-448b-a86e-07e4368d8dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:56.850950 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a01f0f-f8e5-448b-a86e-07e4368d8dc3 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:56.851736 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a01f0f-f8e5-448b-a86e-07e4368d8dc3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:56.852041 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a01f0f-f8e5-448b-a86e-07e4368d8dc3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:56.853687 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a01f0f-f8e5-448b-a86e-07e4368d8dc3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:56.871330 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 540/1072] 10.222.0.40 () {56 vars in 1133 bytes} [Fri Aug 5 10:14:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:14:56.877355 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c73951-f3eb-4360-ae89-fa41aa2b3622 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:56.937256 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:56.984578 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c73951-f3eb-4360-ae89-fa41aa2b3622 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:57.018299 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09407ff2-b10b-4090-8690-5429aa6dcbbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:57.018930 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09407ff2-b10b-4090-8690-5429aa6dcbbf None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:57.019234 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09407ff2-b10b-4090-8690-5429aa6dcbbf None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:57.019503 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09407ff2-b10b-4090-8690-5429aa6dcbbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:57.029875 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09407ff2-b10b-4090-8690-5429aa6dcbbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:57.031342 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09407ff2-b10b-4090-8690-5429aa6dcbbf None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:57.046335 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-09407ff2-b10b-4090-8690-5429aa6dcbbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:57.046335 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:57.046335 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:57.046335 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:57.046335 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:57.046335 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:57.046335 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:57.046335 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:57.046335 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:57.046335 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:57.046335 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:57.046335 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:57.046335 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:14:57.046335 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:14:57.046335 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:14:57.047306 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:14:57.047306 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:57.047306 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:57.047306 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:57.047306 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:57.047306 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:57.047306 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:57.047306 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:57.047306 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:57.047306 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:57.047306 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:57.047306 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:57.047306 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:57.047306 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:57.047306 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:57.047306 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:57.047306 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:57.048290 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:14:57.048290 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:14:57.048290 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:14:57.048290 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:57.048290 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 533/1073] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:14:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:57.053692 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a01f0f-f8e5-448b-a86e-07e4368d8dc3 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:57.144320 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:57.161912 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a01f0f-f8e5-448b-a86e-07e4368d8dc3 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:57.198390 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763654c8-3978-432a-8fcf-b23b7cf7f113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:57.198969 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763654c8-3978-432a-8fcf-b23b7cf7f113 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:57.199243 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763654c8-3978-432a-8fcf-b23b7cf7f113 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:57.199517 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763654c8-3978-432a-8fcf-b23b7cf7f113 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:57.200207 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763654c8-3978-432a-8fcf-b23b7cf7f113 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:57.200509 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763654c8-3978-432a-8fcf-b23b7cf7f113 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:57.201887 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763654c8-3978-432a-8fcf-b23b7cf7f113 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:57.213188 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 541/1074] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:14:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:57.219033 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09407ff2-b10b-4090-8690-5429aa6dcbbf None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:57.323240 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09407ff2-b10b-4090-8690-5429aa6dcbbf None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:57.325606 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:57.359476 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a4dc5-8671-4ced-aacc-56627ad3bb80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:57.360120 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a4dc5-8671-4ced-aacc-56627ad3bb80 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:57.360403 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a4dc5-8671-4ced-aacc-56627ad3bb80 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:57.360676 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a4dc5-8671-4ced-aacc-56627ad3bb80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:57.367352 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72a4dc5-8671-4ced-aacc-56627ad3bb80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:57.368861 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72a4dc5-8671-4ced-aacc-56627ad3bb80 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:57.380422 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-c72a4dc5-8671-4ced-aacc-56627ad3bb80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:57.380422 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:57.380422 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:57.380422 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:57.380422 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:57.380422 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:57.380422 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:57.380422 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:57.380422 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:57.380422 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:57.380422 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:57.380422 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:57.380422 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 10:14:57.380422 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 10:14:57.380422 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 10:14:57.381352 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 10:14:57.381352 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:57.381352 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:57.381352 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:57.381352 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:57.381352 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:57.381352 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:57.381352 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:57.381352 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:57.381352 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:57.381352 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:57.381352 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:57.381352 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:57.381352 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:57.381352 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:57.381352 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:57.381352 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:57.382799 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 10:14:57.382799 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 10:14:57.382799 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 10:14:57.382799 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 10:14:57.382799 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 10:14:57.382799 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 10:14:57.382799 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:14:57.382799 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:57.382799 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 534/1075] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:14:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:14:57.387630 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763654c8-3978-432a-8fcf-b23b7cf7f113 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:57.487585 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763654c8-3978-432a-8fcf-b23b7cf7f113 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:57.522259 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7247e72-f7b0-40ba-9cbf-4cce8060dda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:57.522871 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7247e72-f7b0-40ba-9cbf-4cce8060dda8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:57.523182 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7247e72-f7b0-40ba-9cbf-4cce8060dda8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:57.523455 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7247e72-f7b0-40ba-9cbf-4cce8060dda8 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:57.524200 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7247e72-f7b0-40ba-9cbf-4cce8060dda8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:57.524534 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7247e72-f7b0-40ba-9cbf-4cce8060dda8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:57.526001 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7247e72-f7b0-40ba-9cbf-4cce8060dda8 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:57.537873 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 542/1076] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 10:14:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:57.544389 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a4dc5-8671-4ced-aacc-56627ad3bb80 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:57.643633 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a4dc5-8671-4ced-aacc-56627ad3bb80 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:57.678621 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9962a-f345-4d0f-9a43-336842fe0ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:57.679326 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9962a-f345-4d0f-9a43-336842fe0ea9 None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:57.679603 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9962a-f345-4d0f-9a43-336842fe0ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:57.679876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9962a-f345-4d0f-9a43-336842fe0ea9 None admin] PATH_INFO: `/v3/projects/36b8b505012d4d0bac54845b6799e743/users/b938f1f41db94b0eafe95bc8eeaae175/roles/f10badb1efd340eda8d6358dc6bd9c28` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:57.682250 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b9962a-f345-4d0f-9a43-336842fe0ea9 None admin] RBAC: Authorizing `identity:create_grant(project_id=36b8b505012d4d0bac54845b6799e743, user_id=b938f1f41db94b0eafe95bc8eeaae175, role_id=f10badb1efd340eda8d6358dc6bd9c28)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:57.683715 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b9962a-f345-4d0f-9a43-336842fe0ea9 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:57.698845 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 535/1077] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 10:14:57 2022] PUT /identity/v3/projects/36b8b505012d4d0bac54845b6799e743/users/b938f1f41db94b0eafe95bc8eeaae175/roles/f10badb1efd340eda8d6358dc6bd9c28 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:14:58.192443 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847c1a5-52af-4931-9f6e-f76ea3414c4c None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:58.192961 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847c1a5-52af-4931-9f6e-f76ea3414c4c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:58.192961 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847c1a5-52af-4931-9f6e-f76ea3414c4c None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:58.194353 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 543/1078] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:58.199680 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f5e2c-5f96-4e44-946e-ce099f854eb3 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:58.200126 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f5e2c-5f96-4e44-946e-ce099f854eb3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:58.200211 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f5e2c-5f96-4e44-946e-ce099f854eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:58.218970 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b90f5e2c-5f96-4e44-946e-ce099f854eb3 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:58.220161 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b90f5e2c-5f96-4e44-946e-ce099f854eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:58.237562 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:58.289140 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 536/1079] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:14:58.311598 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4847c1a5-52af-4931-9f6e-f76ea3414c4c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:58.322989 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:58.417034 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4847c1a5-52af-4931-9f6e-f76ea3414c4c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:58.452079 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07acb7ba-479a-4208-aa6a-ae7c5c7ba235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:58.452694 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07acb7ba-479a-4208-aa6a-ae7c5c7ba235 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:58.459439 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07acb7ba-479a-4208-aa6a-ae7c5c7ba235 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:58.459684 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07acb7ba-479a-4208-aa6a-ae7c5c7ba235 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:58.475518 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07acb7ba-479a-4208-aa6a-ae7c5c7ba235 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:14:58.480573 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-07acb7ba-479a-4208-aa6a-ae7c5c7ba235 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:14:58.480573 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:58.480573 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:58.480573 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:58.480573 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:58.480573 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:58.480573 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:58.480573 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:58.480573 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:58.480573 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:58.480573 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:58.480573 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:58.480573 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:14:58.480573 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:14:58.480573 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:14:58.480573 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:14:58.481536 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:14:58.481536 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:14:58.481536 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:14:58.481536 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:14:58.481536 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:58.481536 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:58.481536 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:58.481536 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:58.481536 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:58.481536 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:58.481536 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:58.481536 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:58.481536 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:58.481536 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:58.481536 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:58.481536 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:58.483049 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:58.483049 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:58.483049 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:58.483049 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:58.483049 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:14:58.483049 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:14:58.483049 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:14:58.483049 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:14:58.483049 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:14:58.483049 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:14:58.483049 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:14:58.483049 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:14:58.483049 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 544/1080] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 10:14:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:14:58.488261 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f5e2c-5f96-4e44-946e-ce099f854eb3 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:58.590638 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f5e2c-5f96-4e44-946e-ce099f854eb3 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:58.625208 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e2deb-85b9-4987-a3d5-29aa9b9b728b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:58.625814 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e2deb-85b9-4987-a3d5-29aa9b9b728b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:58.626091 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e2deb-85b9-4987-a3d5-29aa9b9b728b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:58.626355 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e2deb-85b9-4987-a3d5-29aa9b9b728b None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:58.627094 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636e2deb-85b9-4987-a3d5-29aa9b9b728b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:14:58.627392 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636e2deb-85b9-4987-a3d5-29aa9b9b728b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:58.628757 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636e2deb-85b9-4987-a3d5-29aa9b9b728b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:58.636948 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 537/1081] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:14:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:58.643009 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07acb7ba-479a-4208-aa6a-ae7c5c7ba235 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:58.711507 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:58.744573 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07acb7ba-479a-4208-aa6a-ae7c5c7ba235 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:58.778886 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca0a67c-46fb-4876-84e8-9e418e7bb2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:58.779568 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0a67c-46fb-4876-84e8-9e418e7bb2d9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:58.779845 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0a67c-46fb-4876-84e8-9e418e7bb2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:58.780112 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0a67c-46fb-4876-84e8-9e418e7bb2d9 None admin] PATH_INFO: `/v3/users/391d6d36ccfe472a98b07f33e57a897d` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:58.781278 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca0a67c-46fb-4876-84e8-9e418e7bb2d9 None admin] RBAC: Authorizing `identity:get_user(user_id=391d6d36ccfe472a98b07f33e57a897d)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:58.782713 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca0a67c-46fb-4876-84e8-9e418e7bb2d9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:58.784376 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 545/1082] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:14:58 2022] GET /identity/v3/users/391d6d36ccfe472a98b07f33e57a897d => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:58.790436 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e2deb-85b9-4987-a3d5-29aa9b9b728b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:58.811212 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:58.894501 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e2deb-85b9-4987-a3d5-29aa9b9b728b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:58.929428 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccecd50-c90b-4192-9eba-91fcb4d045bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:58.930061 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccecd50-c90b-4192-9eba-91fcb4d045bd None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:58.930333 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccecd50-c90b-4192-9eba-91fcb4d045bd None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:58.930728 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccecd50-c90b-4192-9eba-91fcb4d045bd None admin] PATH_INFO: `/v3/projects/de6e3edce72b44ac83884bb959e47f2b` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:58.931998 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccecd50-c90b-4192-9eba-91fcb4d045bd None admin] RBAC: Authorizing `identity:get_project(project_id=de6e3edce72b44ac83884bb959e47f2b)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:58.933426 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccecd50-c90b-4192-9eba-91fcb4d045bd None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:58.935152 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 538/1083] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:14:58 2022] GET /identity/v3/projects/de6e3edce72b44ac83884bb959e47f2b => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:14:58.941177 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca0a67c-46fb-4876-84e8-9e418e7bb2d9 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:59.042049 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca0a67c-46fb-4876-84e8-9e418e7bb2d9 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:59.076260 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef0ac8-21e2-4637-bf7b-d03e0bd26505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:59.076929 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef0ac8-21e2-4637-bf7b-d03e0bd26505 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:59.077195 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef0ac8-21e2-4637-bf7b-d03e0bd26505 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:59.077466 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef0ac8-21e2-4637-bf7b-d03e0bd26505 None admin] PATH_INFO: `/v3/projects/de6e3edce72b44ac83884bb959e47f2b/users/391d6d36ccfe472a98b07f33e57a897d/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:59.079424 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef0ac8-21e2-4637-bf7b-d03e0bd26505 None admin] RBAC: Authorizing `identity:create_grant(project_id=de6e3edce72b44ac83884bb959e47f2b, user_id=391d6d36ccfe472a98b07f33e57a897d, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:14:59.080862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef0ac8-21e2-4637-bf7b-d03e0bd26505 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:14:59.083949 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:59.097988 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 546/1084] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 10:14:58 2022] PUT /identity/v3/projects/de6e3edce72b44ac83884bb959e47f2b/users/391d6d36ccfe472a98b07f33e57a897d/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:14:59.717151 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf6aed7-d4ba-4964-bc6a-d0f1056981a6 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:59.717730 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf6aed7-d4ba-4964-bc6a-d0f1056981a6 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:59.717730 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf6aed7-d4ba-4964-bc6a-d0f1056981a6 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:59.719216 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 539/1085] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:14:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:14:59.724529 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16c935-b356-40ab-a29d-301f67e7eb48 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:59.724967 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16c935-b356-40ab-a29d-301f67e7eb48 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:59.725073 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16c935-b356-40ab-a29d-301f67e7eb48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:59.744264 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6d16c935-b356-40ab-a29d-301f67e7eb48 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:14:59.745445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6d16c935-b356-40ab-a29d-301f67e7eb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:14:59.815398 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 547/1086] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:14:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:14:59.839233 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf6aed7-d4ba-4964-bc6a-d0f1056981a6 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:14:59.930252 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:14:59.947518 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf6aed7-d4ba-4964-bc6a-d0f1056981a6 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:14:59.980590 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccdab6b-44bf-4830-9aa5-1d6f681e4363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:14:59.981143 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccdab6b-44bf-4830-9aa5-1d6f681e4363 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:14:59.981419 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccdab6b-44bf-4830-9aa5-1d6f681e4363 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:14:59.981684 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccdab6b-44bf-4830-9aa5-1d6f681e4363 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:14:59.991927 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccdab6b-44bf-4830-9aa5-1d6f681e4363 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:14:59.997000 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-7ccdab6b-44bf-4830-9aa5-1d6f681e4363 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 10:14:59.997000 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:14:59.997000 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:14:59.997000 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:14:59.997000 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:14:59.997000 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:14:59.997000 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:14:59.997000 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:14:59.997000 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:14:59.997000 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:14:59.997000 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:14:59.997000 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:14:59.997000 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:14:59.997000 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:14:59.997000 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:14:59.997000 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:14:59.998015 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:14:59.998015 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:14:59.998015 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:14:59.998015 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:14:59.998015 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:14:59.998015 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:14:59.998015 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:14:59.998015 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:14:59.998015 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:14:59.998015 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:14:59.998015 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:14:59.998015 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:14:59.998015 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:14:59.998015 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:14:59.998015 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:14:59.998015 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:14:59.999694 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:14:59.999694 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:14:59.999694 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:14:59.999694 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:14:59.999694 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:14:59.999694 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:14:59.999694 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:14:59.999694 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:14:59.999694 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:14:59.999694 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:14:59.999694 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 10:14:59.999694 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:14:59.999694 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 540/1087] 10.222.0.40 () {56 vars in 1120 bytes} [Fri Aug 5 10:14:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:15:00.004588 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d16c935-b356-40ab-a29d-301f67e7eb48 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:00.103624 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d16c935-b356-40ab-a29d-301f67e7eb48 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:00.129713 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:00.140445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d891d-e8f1-44c3-a98c-f06b2d7850c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:00.141070 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d891d-e8f1-44c3-a98c-f06b2d7850c9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:00.141375 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d891d-e8f1-44c3-a98c-f06b2d7850c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:00.141642 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d891d-e8f1-44c3-a98c-f06b2d7850c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:00.142336 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849d891d-e8f1-44c3-a98c-f06b2d7850c9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:15:00.142633 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849d891d-e8f1-44c3-a98c-f06b2d7850c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:00.144124 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849d891d-e8f1-44c3-a98c-f06b2d7850c9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:00.157206 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 548/1088] 10.222.0.40 () {56 vars in 1115 bytes} [Fri Aug 5 10:15:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:00.158480 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccdab6b-44bf-4830-9aa5-1d6f681e4363 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:00.267907 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccdab6b-44bf-4830-9aa5-1d6f681e4363 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:00.303902 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b591979-6964-4428-aa1b-d1c1cee0d352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:00.304535 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b591979-6964-4428-aa1b-d1c1cee0d352 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:00.304814 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b591979-6964-4428-aa1b-d1c1cee0d352 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:00.305081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b591979-6964-4428-aa1b-d1c1cee0d352 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:00.316364 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b591979-6964-4428-aa1b-d1c1cee0d352 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:00.317812 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b591979-6964-4428-aa1b-d1c1cee0d352 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:00.320378 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:00.333558 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-3b591979-6964-4428-aa1b-d1c1cee0d352 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:15:00.333558 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:15:00.333558 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:15:00.333558 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:15:00.333558 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:15:00.333558 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:15:00.333558 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:15:00.333558 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:15:00.333558 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:15:00.333558 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:15:00.333558 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:15:00.333558 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:15:00.333558 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:15:00.333558 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:15:00.333558 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:15:00.334468 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:15:00.334468 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:15:00.334468 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:15:00.334468 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:15:00.334468 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:15:00.334468 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:15:00.334468 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:15:00.334468 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:15:00.334468 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:15:00.334468 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:15:00.334468 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:15:00.334468 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:15:00.334468 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:15:00.334468 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:15:00.334468 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:15:00.334468 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:15:00.334468 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:15:00.335398 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:15:00.335398 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:15:00.335398 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:15:00.335398 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:15:00.335398 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 541/1089] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:15:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:15:00.341237 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d891d-e8f1-44c3-a98c-f06b2d7850c9 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:00.441528 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d891d-e8f1-44c3-a98c-f06b2d7850c9 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:00.475471 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0230a4c-1387-4708-94c7-0ed070e507d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:00.476080 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0230a4c-1387-4708-94c7-0ed070e507d6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:00.476356 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0230a4c-1387-4708-94c7-0ed070e507d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:00.476624 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0230a4c-1387-4708-94c7-0ed070e507d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:00.477305 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0230a4c-1387-4708-94c7-0ed070e507d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:15:00.477650 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0230a4c-1387-4708-94c7-0ed070e507d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:00.479034 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0230a4c-1387-4708-94c7-0ed070e507d6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:00.490726 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 549/1090] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:15:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:00.496619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b591979-6964-4428-aa1b-d1c1cee0d352 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:00.600302 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b591979-6964-4428-aa1b-d1c1cee0d352 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:00.634099 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ccadb-9eb1-4b52-8627-0d83020011e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:00.634713 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ccadb-9eb1-4b52-8627-0d83020011e1 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:00.635022 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ccadb-9eb1-4b52-8627-0d83020011e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:00.635289 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ccadb-9eb1-4b52-8627-0d83020011e1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:00.642656 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970ccadb-9eb1-4b52-8627-0d83020011e1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:00.644122 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970ccadb-9eb1-4b52-8627-0d83020011e1 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:00.655452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-970ccadb-9eb1-4b52-8627-0d83020011e1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 10:15:00.655452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:15:00.655452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:15:00.655452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:15:00.655452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:15:00.655452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:15:00.655452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:15:00.655452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:15:00.655452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:15:00.655452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:15:00.655452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:15:00.655452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:15:00.655452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 10:15:00.655452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 10:15:00.655452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 10:15:00.655452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 10:15:00.656425 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:15:00.656425 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:15:00.656425 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 10:15:00.656425 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:15:00.656425 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 10:15:00.656425 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:15:00.656425 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:15:00.656425 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:15:00.656425 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:15:00.656425 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:15:00.656425 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:15:00.656425 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:15:00.656425 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:15:00.656425 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:15:00.656425 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:15:00.656425 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:15:00.656425 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:15:00.656425 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:15:00.657356 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:15:00.657356 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:15:00.657356 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 10:15:00.657356 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 10:15:00.657356 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 10:15:00.657356 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 10:15:00.657356 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 10:15:00.657356 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 10:15:00.657356 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 10:15:00.657356 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:15:00.657356 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 542/1091] 10.222.0.40 () {56 vars in 1093 bytes} [Fri Aug 5 10:15:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:15:00.662641 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0230a4c-1387-4708-94c7-0ed070e507d6 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:00.712957 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:00.764975 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0230a4c-1387-4708-94c7-0ed070e507d6 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:00.802205 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55a4676-05f4-4888-9b1f-868d40b4b4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:00.802822 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a4676-05f4-4888-9b1f-868d40b4b4a4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:00.803118 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a4676-05f4-4888-9b1f-868d40b4b4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:00.803388 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a4676-05f4-4888-9b1f-868d40b4b4a4 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:00.804175 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55a4676-05f4-4888-9b1f-868d40b4b4a4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:15:00.804477 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55a4676-05f4-4888-9b1f-868d40b4b4a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:00.805883 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55a4676-05f4-4888-9b1f-868d40b4b4a4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:00.823111 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 550/1092] 10.222.0.40 () {56 vars in 1133 bytes} [Fri Aug 5 10:15:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:00.828948 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ccadb-9eb1-4b52-8627-0d83020011e1 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:00.888156 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:00.932950 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ccadb-9eb1-4b52-8627-0d83020011e1 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:00.967563 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf5411-630e-4da9-9c13-bcfbb1038fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:00.968152 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf5411-630e-4da9-9c13-bcfbb1038fd9 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:00.968426 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf5411-630e-4da9-9c13-bcfbb1038fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:00.968692 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf5411-630e-4da9-9c13-bcfbb1038fd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:01.036744 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baf5411-630e-4da9-9c13-bcfbb1038fd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:01.038210 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baf5411-630e-4da9-9c13-bcfbb1038fd9 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:01.094288 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:01.105226 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0b787-fc2e-4ec3-a1fe-a595c89ec981 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:01.105509 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0b787-fc2e-4ec3-a1fe-a595c89ec981 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:01.105786 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0b787-fc2e-4ec3-a1fe-a595c89ec981 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:01.107177 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 551/1093] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:15:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 10:15:01.111928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b72511-73c7-47fa-8dd0-e7976912d5cb None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:01.112210 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b72511-73c7-47fa-8dd0-e7976912d5cb None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:01.112482 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b72511-73c7-47fa-8dd0-e7976912d5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:01.130602 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f6b72511-73c7-47fa-8dd0-e7976912d5cb None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:15:01.131835 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f6b72511-73c7-47fa-8dd0-e7976912d5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:15:01.141101 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 543/1094] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:15:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 10:15:01.146792 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf5411-630e-4da9-9c13-bcfbb1038fd9 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:01.201747 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 552/1095] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:15:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 10:15:01.227926 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b72511-73c7-47fa-8dd0-e7976912d5cb None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:01.248409 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf5411-630e-4da9-9c13-bcfbb1038fd9 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:01.261612 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:01.282502 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6954c4e-c33d-41fe-a428-6a85a6e2b058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:01.283179 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6954c4e-c33d-41fe-a428-6a85a6e2b058 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:01.283459 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6954c4e-c33d-41fe-a428-6a85a6e2b058 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:01.283753 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6954c4e-c33d-41fe-a428-6a85a6e2b058 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:01.294174 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6954c4e-c33d-41fe-a428-6a85a6e2b058 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:01.295643 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6954c4e-c33d-41fe-a428-6a85a6e2b058 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:01.311385 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-f6954c4e-c33d-41fe-a428-6a85a6e2b058 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:15:01.311385 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:15:01.311385 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:15:01.311385 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:15:01.311385 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:15:01.311385 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:15:01.311385 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:15:01.311385 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:15:01.311385 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:15:01.311385 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:15:01.311385 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:15:01.311385 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:15:01.311385 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:15:01.311385 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:15:01.311385 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:15:01.312340 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:15:01.312340 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:15:01.312340 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:15:01.312340 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:15:01.312340 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:15:01.312340 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:15:01.312340 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:15:01.312340 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:15:01.312340 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:15:01.312340 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:15:01.312340 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:15:01.312340 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:15:01.312340 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:15:01.312340 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:15:01.312340 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:15:01.312340 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:15:01.312340 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:15:01.313329 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:15:01.313329 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:15:01.313329 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:15:01.313329 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:15:01.313329 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 544/1096] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:15:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:15:01.318307 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6954c4e-c33d-41fe-a428-6a85a6e2b058 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:01.327867 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b72511-73c7-47fa-8dd0-e7976912d5cb None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:01.331491 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:01.364836 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f494cc-0d2a-4f40-be12-25df89c79d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:01.365488 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f494cc-0d2a-4f40-be12-25df89c79d71 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:01.365770 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f494cc-0d2a-4f40-be12-25df89c79d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:01.366045 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f494cc-0d2a-4f40-be12-25df89c79d71 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:01.376821 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f494cc-0d2a-4f40-be12-25df89c79d71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:15:01.381679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-47f494cc-0d2a-4f40-be12-25df89c79d71 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:15:01.381679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:15:01.381679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:15:01.381679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:15:01.381679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:15:01.381679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:15:01.381679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:15:01.381679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:15:01.381679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:15:01.381679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:15:01.381679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:15:01.381679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:15:01.381679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:15:01.381679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:15:01.381679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:15:01.381679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:15:01.382629 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:15:01.382629 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:15:01.382629 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:15:01.382629 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:15:01.382629 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:15:01.382629 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:15:01.382629 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:15:01.382629 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:15:01.382629 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:15:01.382629 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:15:01.382629 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:15:01.382629 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:15:01.382629 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:15:01.382629 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:15:01.382629 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:15:01.382629 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:15:01.383906 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:15:01.383906 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:15:01.383906 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:15:01.383906 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:15:01.383906 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:15:01.383906 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:15:01.383906 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:15:01.383906 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:15:01.383906 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:15:01.383906 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:15:01.383906 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:15:01.383906 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:15:01.383906 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 553/1097] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 10:15:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:15:01.388998 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f494cc-0d2a-4f40-be12-25df89c79d71 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:01.420973 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6954c4e-c33d-41fe-a428-6a85a6e2b058 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:01.456403 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a92de42-b9bb-4827-9ffd-08450c831385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:01.457019 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92de42-b9bb-4827-9ffd-08450c831385 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:01.457298 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92de42-b9bb-4827-9ffd-08450c831385 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:01.457568 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92de42-b9bb-4827-9ffd-08450c831385 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:01.458257 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a92de42-b9bb-4827-9ffd-08450c831385 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:15:01.458540 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a92de42-b9bb-4827-9ffd-08450c831385 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:01.459981 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a92de42-b9bb-4827-9ffd-08450c831385 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:01.471629 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 545/1098] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:15:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:01.477303 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a92de42-b9bb-4827-9ffd-08450c831385 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:01.482541 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:01.489134 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f494cc-0d2a-4f40-be12-25df89c79d71 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:01.525153 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3318bb-ed88-4e6b-a93e-e5ad99808286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:01.525782 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3318bb-ed88-4e6b-a93e-e5ad99808286 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:01.526061 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3318bb-ed88-4e6b-a93e-e5ad99808286 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:01.526334 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3318bb-ed88-4e6b-a93e-e5ad99808286 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:01.527077 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3318bb-ed88-4e6b-a93e-e5ad99808286 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:15:01.527388 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3318bb-ed88-4e6b-a93e-e5ad99808286 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:01.528781 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3318bb-ed88-4e6b-a93e-e5ad99808286 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:01.537298 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 554/1099] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:15:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:15:01.542679 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:01.546378 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3318bb-ed88-4e6b-a93e-e5ad99808286 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:01.576332 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a92de42-b9bb-4827-9ffd-08450c831385 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:01.609650 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e343c5-238e-42b3-9857-2f6a9ef18afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:01.610299 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e343c5-238e-42b3-9857-2f6a9ef18afd None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:01.610578 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e343c5-238e-42b3-9857-2f6a9ef18afd None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:01.610852 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e343c5-238e-42b3-9857-2f6a9ef18afd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:01.617578 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e343c5-238e-42b3-9857-2f6a9ef18afd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:01.619088 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e343c5-238e-42b3-9857-2f6a9ef18afd None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:01.630350 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-10e343c5-238e-42b3-9857-2f6a9ef18afd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:15:01.630350 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:15:01.630350 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:15:01.630350 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:15:01.630350 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:15:01.630350 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:15:01.630350 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:15:01.630350 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:15:01.630350 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:15:01.630350 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:15:01.630350 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:15:01.630350 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:15:01.630350 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 10:15:01.630350 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 10:15:01.630350 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 10:15:01.631295 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 10:15:01.631295 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:15:01.631295 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:15:01.631295 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:15:01.631295 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:15:01.631295 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:15:01.631295 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:15:01.631295 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:15:01.631295 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:15:01.631295 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:15:01.631295 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:15:01.631295 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:15:01.631295 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:15:01.631295 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:15:01.631295 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:15:01.631295 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:15:01.631295 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:15:01.632186 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 10:15:01.632186 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 10:15:01.632186 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 10:15:01.632186 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 10:15:01.632186 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 10:15:01.632186 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 10:15:01.632186 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:15:01.632186 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:15:01.635060 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 546/1100] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:15:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:15:01.637323 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e343c5-238e-42b3-9857-2f6a9ef18afd None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:01.645607 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3318bb-ed88-4e6b-a93e-e5ad99808286 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:01.681219 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cce8df4-cb06-41c0-bcd9-761c4b00d7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:01.681784 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce8df4-cb06-41c0-bcd9-761c4b00d7e7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:01.682058 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce8df4-cb06-41c0-bcd9-761c4b00d7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:01.682323 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce8df4-cb06-41c0-bcd9-761c4b00d7e7 None admin] PATH_INFO: `/v3/users/391d6d36ccfe472a98b07f33e57a897d` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:01.683505 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cce8df4-cb06-41c0-bcd9-761c4b00d7e7 None admin] RBAC: Authorizing `identity:get_user(user_id=391d6d36ccfe472a98b07f33e57a897d)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:01.684928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cce8df4-cb06-41c0-bcd9-761c4b00d7e7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:01.686519 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 555/1101] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:15:01 2022] GET /identity/v3/users/391d6d36ccfe472a98b07f33e57a897d => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:15:01.692166 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cce8df4-cb06-41c0-bcd9-761c4b00d7e7 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:01.718248 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:01.736307 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e343c5-238e-42b3-9857-2f6a9ef18afd None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:01.761549 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:01.769587 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b4238-905f-444e-985f-f48f46ecec98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:01.770208 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b4238-905f-444e-985f-f48f46ecec98 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:01.770503 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b4238-905f-444e-985f-f48f46ecec98 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:01.770772 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b4238-905f-444e-985f-f48f46ecec98 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:01.771541 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73b4238-905f-444e-985f-f48f46ecec98 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:15:01.771842 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73b4238-905f-444e-985f-f48f46ecec98 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:01.773273 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73b4238-905f-444e-985f-f48f46ecec98 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:01.785244 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 547/1102] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 10:15:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:01.790926 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cce8df4-cb06-41c0-bcd9-761c4b00d7e7 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:01.791738 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b4238-905f-444e-985f-f48f46ecec98 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:01.826485 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893649b1-a6f9-428a-81c6-6e9cad93c735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:01.827136 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893649b1-a6f9-428a-81c6-6e9cad93c735 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:01.827422 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893649b1-a6f9-428a-81c6-6e9cad93c735 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:01.827697 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893649b1-a6f9-428a-81c6-6e9cad93c735 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:01.888960 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b4238-905f-444e-985f-f48f46ecec98 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:01.896078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893649b1-a6f9-428a-81c6-6e9cad93c735 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:01.897570 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893649b1-a6f9-428a-81c6-6e9cad93c735 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:01.923632 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdd0354-7811-4416-9b6c-ea38a25f8609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:01.923848 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:01.924244 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdd0354-7811-4416-9b6c-ea38a25f8609 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:01.924522 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdd0354-7811-4416-9b6c-ea38a25f8609 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:01.924846 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdd0354-7811-4416-9b6c-ea38a25f8609 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:01.925664 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdd0354-7811-4416-9b6c-ea38a25f8609 None admin] RBAC: Adding query filter params (role.id=f10badb1efd340eda8d6358dc6bd9c28, scope.project.id=36b8b505012d4d0bac54845b6799e743, user.id=b938f1f41db94b0eafe95bc8eeaae175) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:15:01.925984 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdd0354-7811-4416-9b6c-ea38a25f8609 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:01.927586 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdd0354-7811-4416-9b6c-ea38a25f8609 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:01.933479 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 548/1103] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:15:01 2022] GET /identity/v3/role_assignments?user.id=b938f1f41db94b0eafe95bc8eeaae175&scope.project.id=36b8b505012d4d0bac54845b6799e743&role.id=f10badb1efd340eda8d6358dc6bd9c28 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:02.001533 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 556/1104] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:15:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 05 10:15:02.008017 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdd0354-7811-4416-9b6c-ea38a25f8609 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:02.066849 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:02.120771 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdd0354-7811-4416-9b6c-ea38a25f8609 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:02.157101 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc59b2a8-c940-44d6-90be-00e2301083b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:02.157766 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc59b2a8-c940-44d6-90be-00e2301083b4 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:02.158048 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc59b2a8-c940-44d6-90be-00e2301083b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:02.158305 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc59b2a8-c940-44d6-90be-00e2301083b4 None admin] PATH_INFO: `/v3/projects/de6e3edce72b44ac83884bb959e47f2b` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:02.159522 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc59b2a8-c940-44d6-90be-00e2301083b4 None admin] RBAC: Authorizing `identity:get_project(project_id=de6e3edce72b44ac83884bb959e47f2b)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:02.160966 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc59b2a8-c940-44d6-90be-00e2301083b4 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:02.162674 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 549/1105] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:15:02 2022] GET /identity/v3/projects/de6e3edce72b44ac83884bb959e47f2b => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:02.169416 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893649b1-a6f9-428a-81c6-6e9cad93c735 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:02.271611 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893649b1-a6f9-428a-81c6-6e9cad93c735 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:02.298096 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:02.308143 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aeb92f-546d-4d48-8a31-e8fb66053cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:02.308768 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aeb92f-546d-4d48-8a31-e8fb66053cf7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:02.309045 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aeb92f-546d-4d48-8a31-e8fb66053cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:02.309312 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aeb92f-546d-4d48-8a31-e8fb66053cf7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:02.310094 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aeb92f-546d-4d48-8a31-e8fb66053cf7 None admin] RBAC: Adding query filter params (role.id=daef89adcaeb48a382a0bcb03340ccbb, scope.project.id=de6e3edce72b44ac83884bb959e47f2b, user.id=391d6d36ccfe472a98b07f33e57a897d) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:15:02.310398 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aeb92f-546d-4d48-8a31-e8fb66053cf7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:02.311868 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aeb92f-546d-4d48-8a31-e8fb66053cf7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:02.317811 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 557/1106] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:15:02 2022] GET /identity/v3/role_assignments?user.id=391d6d36ccfe472a98b07f33e57a897d&scope.project.id=de6e3edce72b44ac83884bb959e47f2b&role.id=daef89adcaeb48a382a0bcb03340ccbb => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:04.329903 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d7e3b-842b-4dc1-a08d-abc781c60132 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:04.330463 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d7e3b-842b-4dc1-a08d-abc781c60132 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:04.330463 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d7e3b-842b-4dc1-a08d-abc781c60132 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:04.331871 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 550/1107] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:15:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:15:04.338759 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8d9cb-1954-458a-8330-ee6c960aa0d3 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:04.339492 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8d9cb-1954-458a-8330-ee6c960aa0d3 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:04.339610 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8d9cb-1954-458a-8330-ee6c960aa0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:04.368783 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b1f8d9cb-1954-458a-8330-ee6c960aa0d3 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:15:04.369999 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b1f8d9cb-1954-458a-8330-ee6c960aa0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:15:04.439974 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 558/1108] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:15:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:15:04.466747 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982d7e3b-842b-4dc1-a08d-abc781c60132 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:04.577860 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982d7e3b-842b-4dc1-a08d-abc781c60132 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:04.597283 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:04.613471 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe87761-8397-4eac-9d3f-df13ce8f0596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:04.614104 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe87761-8397-4eac-9d3f-df13ce8f0596 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:04.614376 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe87761-8397-4eac-9d3f-df13ce8f0596 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:04.614641 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe87761-8397-4eac-9d3f-df13ce8f0596 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:04.615867 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe87761-8397-4eac-9d3f-df13ce8f0596 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:04.617301 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe87761-8397-4eac-9d3f-df13ce8f0596 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:04.618934 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 551/1109] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:15:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:04.628256 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f8d9cb-1954-458a-8330-ee6c960aa0d3 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:04.737670 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f8d9cb-1954-458a-8330-ee6c960aa0d3 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:04.761403 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:04.777240 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89697777-af4c-4b42-a849-671741284de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:04.777855 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89697777-af4c-4b42-a849-671741284de5 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:04.778135 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89697777-af4c-4b42-a849-671741284de5 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:04.778407 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89697777-af4c-4b42-a849-671741284de5 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:04.779119 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89697777-af4c-4b42-a849-671741284de5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:04.780564 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89697777-af4c-4b42-a849-671741284de5 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:04.804414 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 559/1110] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 5 10:15:04 2022] POST /identity/v3/users => generated 286 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:15:06.840790 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ed849-d08c-4123-be79-8edfd1ff12a9 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:06.841652 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ed849-d08c-4123-be79-8edfd1ff12a9 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:06.841652 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ed849-d08c-4123-be79-8edfd1ff12a9 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:06.843828 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 552/1111] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:15:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:15:06.850110 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90febf71-174a-4188-ac54-33892dcb4485 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:06.850539 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90febf71-174a-4188-ac54-33892dcb4485 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:06.850671 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90febf71-174a-4188-ac54-33892dcb4485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:06.870474 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-90febf71-174a-4188-ac54-33892dcb4485 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:15:06.871709 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-90febf71-174a-4188-ac54-33892dcb4485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:15:06.949990 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 560/1112] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:15:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:15:06.973308 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ed849-d08c-4123-be79-8edfd1ff12a9 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:07.104510 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ed849-d08c-4123-be79-8edfd1ff12a9 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:07.139068 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a75e2-eae6-4eea-a9aa-8f2fcab63fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:07.139700 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a75e2-eae6-4eea-a9aa-8f2fcab63fc7 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:07.139981 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a75e2-eae6-4eea-a9aa-8f2fcab63fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:07.140249 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a75e2-eae6-4eea-a9aa-8f2fcab63fc7 None admin] PATH_INFO: `/v3/roles/57a070b43ad940f1886c69721dd304c2` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:07.141810 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075a75e2-eae6-4eea-a9aa-8f2fcab63fc7 None admin] RBAC: Authorizing `identity:get_role(role_id=57a070b43ad940f1886c69721dd304c2)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:07.143212 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075a75e2-eae6-4eea-a9aa-8f2fcab63fc7 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:07.144583 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 553/1113] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:15:06 2022] GET /identity/v3/roles/57a070b43ad940f1886c69721dd304c2 => generated 223 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:07.151092 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90febf71-174a-4188-ac54-33892dcb4485 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:07.277431 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90febf71-174a-4188-ac54-33892dcb4485 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:07.285146 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:07.317882 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c9dbb-462d-4227-b4dd-4723e5c00385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:07.318526 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c9dbb-462d-4227-b4dd-4723e5c00385 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:07.318808 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c9dbb-462d-4227-b4dd-4723e5c00385 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:07.319116 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c9dbb-462d-4227-b4dd-4723e5c00385 None admin] PATH_INFO: `/v3/users/c10f25e95a4547baaabb3070d3ee1a51` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:07.332708 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c9dbb-462d-4227-b4dd-4723e5c00385 None admin] RBAC: Authorizing `identity:get_user(user_id=c10f25e95a4547baaabb3070d3ee1a51)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:07.334169 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c9dbb-462d-4227-b4dd-4723e5c00385 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:07.335852 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 561/1114] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:15:07 2022] GET /identity/v3/users/c10f25e95a4547baaabb3070d3ee1a51 => generated 286 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:07.342811 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a75e2-eae6-4eea-a9aa-8f2fcab63fc7 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:07.412526 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:07.479203 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a75e2-eae6-4eea-a9aa-8f2fcab63fc7 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:07.518289 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d38da-b882-4fb7-b160-806d54a335f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:07.518948 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d38da-b882-4fb7-b160-806d54a335f3 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:07.519222 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d38da-b882-4fb7-b160-806d54a335f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:07.519494 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d38da-b882-4fb7-b160-806d54a335f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:07.598296 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d38da-b882-4fb7-b160-806d54a335f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:07.599826 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d38da-b882-4fb7-b160-806d54a335f3 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:07.660718 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:07.707829 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 554/1115] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:15:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 10:15:07.714523 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c9dbb-462d-4227-b4dd-4723e5c00385 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:07.825916 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c9dbb-462d-4227-b4dd-4723e5c00385 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:07.869277 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cc813b-0c1f-43cd-8ec8-ed8d607be660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:07.869989 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc813b-0c1f-43cd-8ec8-ed8d607be660 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:07.870177 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc813b-0c1f-43cd-8ec8-ed8d607be660 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:07.870438 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc813b-0c1f-43cd-8ec8-ed8d607be660 None admin] PATH_INFO: `/v3/projects/de6e3edce72b44ac83884bb959e47f2b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:07.871665 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cc813b-0c1f-43cd-8ec8-ed8d607be660 None admin] RBAC: Authorizing `identity:get_project(project_id=de6e3edce72b44ac83884bb959e47f2b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:07.873099 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cc813b-0c1f-43cd-8ec8-ed8d607be660 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:07.874782 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 562/1116] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:15:07 2022] GET /identity/v3/projects/de6e3edce72b44ac83884bb959e47f2b => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:07.881411 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d38da-b882-4fb7-b160-806d54a335f3 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:07.991809 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d38da-b882-4fb7-b160-806d54a335f3 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:08.034547 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59b450b-7132-4ff6-9429-10656dafcb0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:08.035216 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b450b-7132-4ff6-9429-10656dafcb0e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:08.035492 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b450b-7132-4ff6-9429-10656dafcb0e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:08.035760 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b450b-7132-4ff6-9429-10656dafcb0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:08.036543 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59b450b-7132-4ff6-9429-10656dafcb0e None admin] RBAC: Adding query filter params (role.id=57a070b43ad940f1886c69721dd304c2, scope.project.id=de6e3edce72b44ac83884bb959e47f2b, user.id=c10f25e95a4547baaabb3070d3ee1a51) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:15:08.036842 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59b450b-7132-4ff6-9429-10656dafcb0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:08.038324 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59b450b-7132-4ff6-9429-10656dafcb0e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:08.045459 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 555/1117] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:15:07 2022] GET /identity/v3/role_assignments?user.id=c10f25e95a4547baaabb3070d3ee1a51&scope.project.id=de6e3edce72b44ac83884bb959e47f2b&role.id=57a070b43ad940f1886c69721dd304c2 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:10.026399 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12ff24-4489-439a-9a26-4f9cf2f466d4 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:10.027004 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12ff24-4489-439a-9a26-4f9cf2f466d4 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:10.027004 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12ff24-4489-439a-9a26-4f9cf2f466d4 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:10.028334 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 563/1118] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:15:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:15:10.034604 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1511ffa-758d-48fe-b4ee-b3bac381666f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:10.035356 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1511ffa-758d-48fe-b4ee-b3bac381666f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:10.035565 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1511ffa-758d-48fe-b4ee-b3bac381666f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:10.062726 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a1511ffa-758d-48fe-b4ee-b3bac381666f None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:15:10.064027 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a1511ffa-758d-48fe-b4ee-b3bac381666f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:15:10.079218 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:10.137793 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 556/1119] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:15:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:15:10.160229 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f12ff24-4489-439a-9a26-4f9cf2f466d4 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:10.193500 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:10.265727 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f12ff24-4489-439a-9a26-4f9cf2f466d4 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:10.302745 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbb7fb0-78fd-4cd8-acd3-887d3f0c545c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:10.303332 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb7fb0-78fd-4cd8-acd3-887d3f0c545c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:10.303610 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb7fb0-78fd-4cd8-acd3-887d3f0c545c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:10.303872 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb7fb0-78fd-4cd8-acd3-887d3f0c545c None admin] PATH_INFO: `/v3/roles/57a070b43ad940f1886c69721dd304c2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:10.305655 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbb7fb0-78fd-4cd8-acd3-887d3f0c545c None admin] RBAC: Authorizing `identity:get_role(role_id=57a070b43ad940f1886c69721dd304c2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:10.307053 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbb7fb0-78fd-4cd8-acd3-887d3f0c545c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:10.308389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 564/1120] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:15:10 2022] GET /identity/v3/roles/57a070b43ad940f1886c69721dd304c2 => generated 223 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:10.314963 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1511ffa-758d-48fe-b4ee-b3bac381666f None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:10.418832 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1511ffa-758d-48fe-b4ee-b3bac381666f None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:10.456527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f818a1f-0d5f-40ee-b896-7c49a1495fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:10.457154 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f818a1f-0d5f-40ee-b896-7c49a1495fd5 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:10.457429 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f818a1f-0d5f-40ee-b896-7c49a1495fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:10.457688 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f818a1f-0d5f-40ee-b896-7c49a1495fd5 None admin] PATH_INFO: `/v3/users/c10f25e95a4547baaabb3070d3ee1a51` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:10.458846 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f818a1f-0d5f-40ee-b896-7c49a1495fd5 None admin] RBAC: Authorizing `identity:get_user(user_id=c10f25e95a4547baaabb3070d3ee1a51)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:10.460324 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f818a1f-0d5f-40ee-b896-7c49a1495fd5 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:10.462006 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 557/1121] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:15:10 2022] GET /identity/v3/users/c10f25e95a4547baaabb3070d3ee1a51 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:10.467969 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbb7fb0-78fd-4cd8-acd3-887d3f0c545c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:10.573880 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:10.577172 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbb7fb0-78fd-4cd8-acd3-887d3f0c545c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:10.613004 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e7e448-ee8c-4f69-8ad7-15aba9b229cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:10.613635 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e7e448-ee8c-4f69-8ad7-15aba9b229cb None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:10.613915 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e7e448-ee8c-4f69-8ad7-15aba9b229cb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:10.614185 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e7e448-ee8c-4f69-8ad7-15aba9b229cb None admin] PATH_INFO: `/v3/projects/de6e3edce72b44ac83884bb959e47f2b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:10.615412 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e7e448-ee8c-4f69-8ad7-15aba9b229cb None admin] RBAC: Authorizing `identity:get_project(project_id=de6e3edce72b44ac83884bb959e47f2b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:10.616854 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e7e448-ee8c-4f69-8ad7-15aba9b229cb None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:10.618805 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 565/1122] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:15:10 2022] GET /identity/v3/projects/de6e3edce72b44ac83884bb959e47f2b => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:10.627427 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f818a1f-0d5f-40ee-b896-7c49a1495fd5 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:10.649606 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:10.747178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f818a1f-0d5f-40ee-b896-7c49a1495fd5 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:10.785314 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2992345c-f91f-44a3-af2f-e0d4af368596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:10.785964 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2992345c-f91f-44a3-af2f-e0d4af368596 None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:10.786239 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2992345c-f91f-44a3-af2f-e0d4af368596 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:10.786514 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2992345c-f91f-44a3-af2f-e0d4af368596 None admin] PATH_INFO: `/v3/projects/de6e3edce72b44ac83884bb959e47f2b/users/c10f25e95a4547baaabb3070d3ee1a51/roles/57a070b43ad940f1886c69721dd304c2` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:10.788841 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2992345c-f91f-44a3-af2f-e0d4af368596 None admin] RBAC: Authorizing `identity:create_grant(project_id=de6e3edce72b44ac83884bb959e47f2b, user_id=c10f25e95a4547baaabb3070d3ee1a51, role_id=57a070b43ad940f1886c69721dd304c2)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:10.791158 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2992345c-f91f-44a3-af2f-e0d4af368596 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:10.808702 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 558/1123] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 10:15:10 2022] PUT /identity/v3/projects/de6e3edce72b44ac83884bb959e47f2b/users/c10f25e95a4547baaabb3070d3ee1a51/roles/57a070b43ad940f1886c69721dd304c2 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:15:12.777300 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de690f69-371b-403e-8491-62abd562aa3b None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:12.777989 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de690f69-371b-403e-8491-62abd562aa3b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:12.777989 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de690f69-371b-403e-8491-62abd562aa3b None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:12.779257 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 566/1124] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:15:12.785511 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f9067-73c7-487d-934b-e28304c6c6fc None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:12.786270 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f9067-73c7-487d-934b-e28304c6c6fc None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:12.786464 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f9067-73c7-487d-934b-e28304c6c6fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:12.812658 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ab9f9067-73c7-487d-934b-e28304c6c6fc None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:15:12.813869 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ab9f9067-73c7-487d-934b-e28304c6c6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:15:12.889145 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 559/1125] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:15:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:15:12.912780 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de690f69-371b-403e-8491-62abd562aa3b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:13.020069 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de690f69-371b-403e-8491-62abd562aa3b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:13.055907 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ab1c8-de1f-405c-9bc9-d58a6314b3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:13.056462 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ab1c8-de1f-405c-9bc9-d58a6314b3c1 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:13.056740 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ab1c8-de1f-405c-9bc9-d58a6314b3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:13.057014 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ab1c8-de1f-405c-9bc9-d58a6314b3c1 None admin] PATH_INFO: `/v3/roles/57a070b43ad940f1886c69721dd304c2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:13.058572 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2ab1c8-de1f-405c-9bc9-d58a6314b3c1 None admin] RBAC: Authorizing `identity:get_role(role_id=57a070b43ad940f1886c69721dd304c2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:13.059981 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2ab1c8-de1f-405c-9bc9-d58a6314b3c1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:13.061334 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 567/1126] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:15:12 2022] GET /identity/v3/roles/57a070b43ad940f1886c69721dd304c2 => generated 223 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:13.067922 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9f9067-73c7-487d-934b-e28304c6c6fc None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:13.079943 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:13.181793 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9f9067-73c7-487d-934b-e28304c6c6fc None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:13.218102 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b6bb04-4181-496f-a629-8d5a0375f7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:13.218758 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6bb04-4181-496f-a629-8d5a0375f7d9 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:13.219061 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6bb04-4181-496f-a629-8d5a0375f7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:13.219304 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6bb04-4181-496f-a629-8d5a0375f7d9 None admin] PATH_INFO: `/v3/users/c10f25e95a4547baaabb3070d3ee1a51` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:13.220525 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b6bb04-4181-496f-a629-8d5a0375f7d9 None admin] RBAC: Authorizing `identity:get_user(user_id=c10f25e95a4547baaabb3070d3ee1a51)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:13.221964 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b6bb04-4181-496f-a629-8d5a0375f7d9 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:13.223613 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 560/1127] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:15:13 2022] GET /identity/v3/users/c10f25e95a4547baaabb3070d3ee1a51 => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:13.229977 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ab1c8-de1f-405c-9bc9-d58a6314b3c1 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:13.266391 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:13.334328 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ab1c8-de1f-405c-9bc9-d58a6314b3c1 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:13.369549 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4febc706-e04c-457f-858c-9bfc7b945fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:13.370157 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4febc706-e04c-457f-858c-9bfc7b945fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:13.370432 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4febc706-e04c-457f-858c-9bfc7b945fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:13.370702 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4febc706-e04c-457f-858c-9bfc7b945fe6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:13.439895 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4febc706-e04c-457f-858c-9bfc7b945fe6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:13.441335 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4febc706-e04c-457f-858c-9bfc7b945fe6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:13.475395 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:13.543573 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 568/1128] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:15:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 10:15:13.549823 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b6bb04-4181-496f-a629-8d5a0375f7d9 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:13.634471 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:13.657970 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b6bb04-4181-496f-a629-8d5a0375f7d9 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:13.695936 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e996eb2-e47a-4843-b3cd-498f0487a2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:13.696577 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e996eb2-e47a-4843-b3cd-498f0487a2d0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:13.696842 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e996eb2-e47a-4843-b3cd-498f0487a2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:13.697133 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e996eb2-e47a-4843-b3cd-498f0487a2d0 None admin] PATH_INFO: `/v3/projects/de6e3edce72b44ac83884bb959e47f2b` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:13.698411 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e996eb2-e47a-4843-b3cd-498f0487a2d0 None admin] RBAC: Authorizing `identity:get_project(project_id=de6e3edce72b44ac83884bb959e47f2b)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:13.699906 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e996eb2-e47a-4843-b3cd-498f0487a2d0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:13.701609 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 561/1129] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:15:13 2022] GET /identity/v3/projects/de6e3edce72b44ac83884bb959e47f2b => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:15:13.708274 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4febc706-e04c-457f-858c-9bfc7b945fe6 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:13.810382 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4febc706-e04c-457f-858c-9bfc7b945fe6 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:13.848881 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74245985-12b5-480b-a54f-12f5d8118ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:13.849494 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74245985-12b5-480b-a54f-12f5d8118ed9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:13.849769 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74245985-12b5-480b-a54f-12f5d8118ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:13.850041 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74245985-12b5-480b-a54f-12f5d8118ed9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:13.850826 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74245985-12b5-480b-a54f-12f5d8118ed9 None admin] RBAC: Adding query filter params (role.id=57a070b43ad940f1886c69721dd304c2, scope.project.id=de6e3edce72b44ac83884bb959e47f2b, user.id=c10f25e95a4547baaabb3070d3ee1a51) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:15:13.851170 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74245985-12b5-480b-a54f-12f5d8118ed9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:13.852639 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74245985-12b5-480b-a54f-12f5d8118ed9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:13.855627 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:13.859146 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 569/1130] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:15:13 2022] GET /identity/v3/role_assignments?user.id=c10f25e95a4547baaabb3070d3ee1a51&scope.project.id=de6e3edce72b44ac83884bb959e47f2b&role.id=57a070b43ad940f1886c69721dd304c2 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:15.855568 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427fbedd-9d91-40c4-ae68-d6b819bc44b6 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:15.856124 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427fbedd-9d91-40c4-ae68-d6b819bc44b6 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:15.856124 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427fbedd-9d91-40c4-ae68-d6b819bc44b6 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:15.857734 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 562/1131] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:15:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:15:15.862947 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d9a8b-2b65-472a-9243-5f562bd84492 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:15.863344 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d9a8b-2b65-472a-9243-5f562bd84492 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:15.863521 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d9a8b-2b65-472a-9243-5f562bd84492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:15.881951 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e01d9a8b-2b65-472a-9243-5f562bd84492 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:15:15.883267 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e01d9a8b-2b65-472a-9243-5f562bd84492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:15:15.954448 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 570/1132] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:15:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:15:15.977722 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427fbedd-9d91-40c4-ae68-d6b819bc44b6 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:16.086106 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427fbedd-9d91-40c4-ae68-d6b819bc44b6 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:16.121265 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad6908f-9468-4a2e-abdc-6a369b14c38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:16.125085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad6908f-9468-4a2e-abdc-6a369b14c38d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:16.125085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad6908f-9468-4a2e-abdc-6a369b14c38d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:16.125085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad6908f-9468-4a2e-abdc-6a369b14c38d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:16.125085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad6908f-9468-4a2e-abdc-6a369b14c38d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:16.125430 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad6908f-9468-4a2e-abdc-6a369b14c38d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:16.126802 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 563/1133] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:15:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:16.133264 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01d9a8b-2b65-472a-9243-5f562bd84492 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:16.237677 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01d9a8b-2b65-472a-9243-5f562bd84492 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:16.266825 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:16.274008 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605bae4e-98f4-4b30-9582-1c2b57b37642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:16.274625 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605bae4e-98f4-4b30-9582-1c2b57b37642 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:16.274893 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605bae4e-98f4-4b30-9582-1c2b57b37642 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:16.275199 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605bae4e-98f4-4b30-9582-1c2b57b37642 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:16.275859 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605bae4e-98f4-4b30-9582-1c2b57b37642 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:16.277288 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605bae4e-98f4-4b30-9582-1c2b57b37642 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:16.307048 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 571/1134] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 5 10:15:16 2022] POST /identity/v3/projects => generated 311 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:15:18.311912 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf26da-594d-4c04-8488-40489f58c209 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:18.312486 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf26da-594d-4c04-8488-40489f58c209 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:18.312486 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf26da-594d-4c04-8488-40489f58c209 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:18.313841 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 564/1135] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:15:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:15:18.320503 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d2db4-8837-4840-a422-6db66f0d5229 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:18.320961 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d2db4-8837-4840-a422-6db66f0d5229 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:18.321060 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d2db4-8837-4840-a422-6db66f0d5229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:18.340551 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-5a7d2db4-8837-4840-a422-6db66f0d5229 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:15:18.341759 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5a7d2db4-8837-4840-a422-6db66f0d5229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:15:18.414337 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 572/1136] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:15:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:15:18.451088 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaf26da-594d-4c04-8488-40489f58c209 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:18.468626 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:18.563788 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaf26da-594d-4c04-8488-40489f58c209 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:18.598922 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb1f58-de7e-4d84-b661-de6cfc84c961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:18.599559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb1f58-de7e-4d84-b661-de6cfc84c961 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:18.599845 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb1f58-de7e-4d84-b661-de6cfc84c961 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:18.600111 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb1f58-de7e-4d84-b661-de6cfc84c961 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:18.601354 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdb1f58-de7e-4d84-b661-de6cfc84c961 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:18.602944 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdb1f58-de7e-4d84-b661-de6cfc84c961 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:18.604585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 565/1137] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:15:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:18.611479 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7d2db4-8837-4840-a422-6db66f0d5229 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:18.728009 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7d2db4-8837-4840-a422-6db66f0d5229 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:18.741058 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:18.765453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c27ae9-9063-4917-8dd7-f542d32f5ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:18.766069 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c27ae9-9063-4917-8dd7-f542d32f5ee0 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:18.766344 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c27ae9-9063-4917-8dd7-f542d32f5ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:18.766611 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c27ae9-9063-4917-8dd7-f542d32f5ee0 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:18.767316 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c27ae9-9063-4917-8dd7-f542d32f5ee0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:18.768757 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c27ae9-9063-4917-8dd7-f542d32f5ee0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:18.792383 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 573/1138] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 5 10:15:18 2022] POST /identity/v3/users => generated 286 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:15:20.827668 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c94986-6318-4497-9f03-2413244224ed None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:20.828561 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c94986-6318-4497-9f03-2413244224ed None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:20.828561 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c94986-6318-4497-9f03-2413244224ed None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:20.830671 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 566/1139] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:15:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:15:20.839820 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63057c-bf4c-42c3-ad91-10462b7e4c08 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:20.840237 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63057c-bf4c-42c3-ad91-10462b7e4c08 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:20.840417 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63057c-bf4c-42c3-ad91-10462b7e4c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:20.869678 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6c63057c-bf4c-42c3-ad91-10462b7e4c08 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:15:20.871489 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6c63057c-bf4c-42c3-ad91-10462b7e4c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:15:20.963633 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 574/1140] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:15:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:15:20.997265 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c94986-6318-4497-9f03-2413244224ed None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:21.093753 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:21.114419 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c94986-6318-4497-9f03-2413244224ed None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:21.150310 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c796d5-29fc-464f-b6aa-506725f78670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:21.150947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c796d5-29fc-464f-b6aa-506725f78670 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:21.151236 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c796d5-29fc-464f-b6aa-506725f78670 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:21.151512 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c796d5-29fc-464f-b6aa-506725f78670 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:21.162101 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c796d5-29fc-464f-b6aa-506725f78670 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:15:21.167403 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-99c796d5-29fc-464f-b6aa-506725f78670 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:15:21.167403 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:15:21.167403 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:15:21.167403 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:15:21.167403 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:15:21.167403 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:15:21.167403 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:15:21.167403 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:15:21.167403 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:15:21.167403 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:15:21.167403 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:15:21.167403 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:15:21.167403 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:15:21.167403 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:15:21.167403 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:15:21.167403 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:15:21.168460 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:15:21.168460 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:15:21.168460 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:15:21.168460 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:15:21.168460 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:15:21.168460 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:15:21.168460 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:15:21.168460 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:15:21.168460 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:15:21.168460 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:15:21.168460 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:15:21.168460 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:15:21.168460 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:15:21.168460 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:15:21.168460 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:15:21.168460 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:15:21.169524 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:15:21.169524 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:15:21.169524 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:15:21.169524 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:15:21.169524 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:15:21.169524 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:15:21.169524 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:15:21.169524 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:15:21.169524 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:15:21.169524 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:15:21.169524 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:15:21.169524 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:15:21.169524 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 567/1141] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 10:15:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:15:21.175716 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c63057c-bf4c-42c3-ad91-10462b7e4c08 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:21.276171 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:21.280545 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c63057c-bf4c-42c3-ad91-10462b7e4c08 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:21.317364 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03913b4d-e4e1-45d2-b627-dade51408795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:21.317981 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03913b4d-e4e1-45d2-b627-dade51408795 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:21.318258 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03913b4d-e4e1-45d2-b627-dade51408795 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:21.318528 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03913b4d-e4e1-45d2-b627-dade51408795 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:21.319271 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03913b4d-e4e1-45d2-b627-dade51408795 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:15:21.319637 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03913b4d-e4e1-45d2-b627-dade51408795 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:21.321494 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03913b4d-e4e1-45d2-b627-dade51408795 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:21.329300 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 575/1142] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:15:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:21.336487 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c796d5-29fc-464f-b6aa-506725f78670 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:21.448708 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c796d5-29fc-464f-b6aa-506725f78670 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:21.484361 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6fddd-baaf-446f-a852-fb24876e9fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:21.484997 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6fddd-baaf-446f-a852-fb24876e9fd3 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:21.485291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6fddd-baaf-446f-a852-fb24876e9fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:21.485558 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6fddd-baaf-446f-a852-fb24876e9fd3 None admin] PATH_INFO: `/v3/users/a983d4a108c04e94aa9666f614e0f4fe` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:21.499226 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:21.503492 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d6fddd-baaf-446f-a852-fb24876e9fd3 None admin] RBAC: Authorizing `identity:get_user(user_id=a983d4a108c04e94aa9666f614e0f4fe)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:21.505122 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d6fddd-baaf-446f-a852-fb24876e9fd3 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:21.506792 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 568/1143] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:15:21 2022] GET /identity/v3/users/a983d4a108c04e94aa9666f614e0f4fe => generated 286 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:21.512955 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03913b4d-e4e1-45d2-b627-dade51408795 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:21.630013 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03913b4d-e4e1-45d2-b627-dade51408795 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:21.669400 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7a442-d52b-430d-aa2d-774e90c3a2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:21.670003 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7a442-d52b-430d-aa2d-774e90c3a2aa None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:21.670276 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7a442-d52b-430d-aa2d-774e90c3a2aa None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:21.670542 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7a442-d52b-430d-aa2d-774e90c3a2aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:21.709454 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:21.759054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e7a442-d52b-430d-aa2d-774e90c3a2aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:21.760522 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e7a442-d52b-430d-aa2d-774e90c3a2aa None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:21.862620 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 576/1144] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:15:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 10:15:21.869376 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6fddd-baaf-446f-a852-fb24876e9fd3 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:21.985074 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6fddd-baaf-446f-a852-fb24876e9fd3 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:22.025149 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daae1467-fab3-465d-841a-fa670426f1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:22.025788 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae1467-fab3-465d-841a-fa670426f1ee None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:22.026067 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae1467-fab3-465d-841a-fa670426f1ee None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:22.026337 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae1467-fab3-465d-841a-fa670426f1ee None admin] PATH_INFO: `/v3/projects/316cd09f851e41ae8252ddc4c39fc803` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:22.027642 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daae1467-fab3-465d-841a-fa670426f1ee None admin] RBAC: Authorizing `identity:get_project(project_id=316cd09f851e41ae8252ddc4c39fc803)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:22.029138 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daae1467-fab3-465d-841a-fa670426f1ee None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:22.030812 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 569/1145] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:15:21 2022] GET /identity/v3/projects/316cd09f851e41ae8252ddc4c39fc803 => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:22.037455 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7a442-d52b-430d-aa2d-774e90c3a2aa None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:22.097312 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:22.142457 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7a442-d52b-430d-aa2d-774e90c3a2aa None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:22.182264 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37df8d8-53ff-41d3-9c00-9e7ceac59c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:22.182874 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37df8d8-53ff-41d3-9c00-9e7ceac59c5a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:22.183189 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37df8d8-53ff-41d3-9c00-9e7ceac59c5a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:22.183465 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37df8d8-53ff-41d3-9c00-9e7ceac59c5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:22.184241 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37df8d8-53ff-41d3-9c00-9e7ceac59c5a None admin] RBAC: Adding query filter params (role.id=daef89adcaeb48a382a0bcb03340ccbb, scope.project.id=316cd09f851e41ae8252ddc4c39fc803, user.id=a983d4a108c04e94aa9666f614e0f4fe) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:15:22.184556 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37df8d8-53ff-41d3-9c00-9e7ceac59c5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:22.185999 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37df8d8-53ff-41d3-9c00-9e7ceac59c5a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:22.191591 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 577/1146] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:15:22 2022] GET /identity/v3/role_assignments?user.id=a983d4a108c04e94aa9666f614e0f4fe&scope.project.id=316cd09f851e41ae8252ddc4c39fc803&role.id=daef89adcaeb48a382a0bcb03340ccbb => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:24.140667 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5827a-a40e-41b2-ae83-777a19612570 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:24.141253 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5827a-a40e-41b2-ae83-777a19612570 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:24.141253 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5827a-a40e-41b2-ae83-777a19612570 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:24.143392 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 570/1147] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:15:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:15:24.152244 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2db84e-29ff-4473-b02f-48391fb0aa9d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:24.152660 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2db84e-29ff-4473-b02f-48391fb0aa9d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:24.152949 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2db84e-29ff-4473-b02f-48391fb0aa9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:24.174333 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-da2db84e-29ff-4473-b02f-48391fb0aa9d None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:15:24.175572 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-da2db84e-29ff-4473-b02f-48391fb0aa9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:15:24.244687 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 578/1148] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:15:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:15:24.267520 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb5827a-a40e-41b2-ae83-777a19612570 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:24.349105 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:24.385940 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb5827a-a40e-41b2-ae83-777a19612570 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:24.420599 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d34e7b1-f216-42a2-9813-2f2cdd6639af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:24.421164 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34e7b1-f216-42a2-9813-2f2cdd6639af None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:24.421438 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34e7b1-f216-42a2-9813-2f2cdd6639af None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:24.421702 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34e7b1-f216-42a2-9813-2f2cdd6639af None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:24.432352 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d34e7b1-f216-42a2-9813-2f2cdd6639af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:15:24.437178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-0d34e7b1-f216-42a2-9813-2f2cdd6639af None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:15:24.437178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:15:24.437178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:15:24.437178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:15:24.437178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:15:24.437178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:15:24.437178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:15:24.437178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:15:24.437178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:15:24.437178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:15:24.437178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:15:24.437178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:15:24.437178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:15:24.437178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:15:24.437178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:15:24.437178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:15:24.438273 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:15:24.438273 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:15:24.438273 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:15:24.438273 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:15:24.438273 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:15:24.438273 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:15:24.438273 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:15:24.438273 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:15:24.438273 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:15:24.438273 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:15:24.438273 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:15:24.438273 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:15:24.438273 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:15:24.438273 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:15:24.438273 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:15:24.438273 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:15:24.439155 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:15:24.439155 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:15:24.439155 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:15:24.439155 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:15:24.439155 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:15:24.439155 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:15:24.439155 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:15:24.439155 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:15:24.439155 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:15:24.439155 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:15:24.439155 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:15:24.439155 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:15:24.439155 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 571/1149] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 10:15:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:15:24.447140 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2db84e-29ff-4473-b02f-48391fb0aa9d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:24.509651 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:24.557625 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2db84e-29ff-4473-b02f-48391fb0aa9d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:24.605733 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e02e609-bd41-4f70-baf0-ffb3cf843c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:24.606345 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e02e609-bd41-4f70-baf0-ffb3cf843c0e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:24.606618 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e02e609-bd41-4f70-baf0-ffb3cf843c0e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:24.606886 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e02e609-bd41-4f70-baf0-ffb3cf843c0e None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:24.607621 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e02e609-bd41-4f70-baf0-ffb3cf843c0e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:15:24.607922 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e02e609-bd41-4f70-baf0-ffb3cf843c0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:24.609299 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e02e609-bd41-4f70-baf0-ffb3cf843c0e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:24.617800 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 579/1150] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:15:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:24.627238 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d34e7b1-f216-42a2-9813-2f2cdd6639af None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:24.739447 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d34e7b1-f216-42a2-9813-2f2cdd6639af None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:24.765607 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:24.776031 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e10a4-4db4-4c33-a50d-ebb708844a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:24.776665 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e10a4-4db4-4c33-a50d-ebb708844a11 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:24.776944 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e10a4-4db4-4c33-a50d-ebb708844a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:24.777216 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e10a4-4db4-4c33-a50d-ebb708844a11 None admin] PATH_INFO: `/v3/users/a983d4a108c04e94aa9666f614e0f4fe` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:24.778436 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536e10a4-4db4-4c33-a50d-ebb708844a11 None admin] RBAC: Authorizing `identity:get_user(user_id=a983d4a108c04e94aa9666f614e0f4fe)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:24.779921 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536e10a4-4db4-4c33-a50d-ebb708844a11 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:24.781548 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 572/1151] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:15:24 2022] GET /identity/v3/users/a983d4a108c04e94aa9666f614e0f4fe => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:24.788420 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e02e609-bd41-4f70-baf0-ffb3cf843c0e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:24.896308 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e02e609-bd41-4f70-baf0-ffb3cf843c0e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:24.912464 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:24.935199 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac8a961-e42a-4096-bbde-df5b89f9ef3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:24.936074 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8a961-e42a-4096-bbde-df5b89f9ef3a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:24.936405 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8a961-e42a-4096-bbde-df5b89f9ef3a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:24.936762 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8a961-e42a-4096-bbde-df5b89f9ef3a None admin] PATH_INFO: `/v3/projects/316cd09f851e41ae8252ddc4c39fc803` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:24.938592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac8a961-e42a-4096-bbde-df5b89f9ef3a None admin] RBAC: Authorizing `identity:get_project(project_id=316cd09f851e41ae8252ddc4c39fc803)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:24.940996 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac8a961-e42a-4096-bbde-df5b89f9ef3a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:24.943516 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 580/1152] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:15:24 2022] GET /identity/v3/projects/316cd09f851e41ae8252ddc4c39fc803 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:24.950314 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e10a4-4db4-4c33-a50d-ebb708844a11 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:25.074737 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e10a4-4db4-4c33-a50d-ebb708844a11 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:25.113120 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a4bd51-470c-4717-9b55-d84572957c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:25.113776 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4bd51-470c-4717-9b55-d84572957c4a None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:25.114052 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4bd51-470c-4717-9b55-d84572957c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:25.114320 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4bd51-470c-4717-9b55-d84572957c4a None admin] PATH_INFO: `/v3/projects/316cd09f851e41ae8252ddc4c39fc803/users/a983d4a108c04e94aa9666f614e0f4fe/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:25.116611 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a4bd51-470c-4717-9b55-d84572957c4a None admin] RBAC: Authorizing `identity:create_grant(project_id=316cd09f851e41ae8252ddc4c39fc803, user_id=a983d4a108c04e94aa9666f614e0f4fe, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:25.118147 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a4bd51-470c-4717-9b55-d84572957c4a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:25.135823 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 573/1153] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 10:15:24 2022] PUT /identity/v3/projects/316cd09f851e41ae8252ddc4c39fc803/users/a983d4a108c04e94aa9666f614e0f4fe/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:15:27.103275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76ea3f-3bdd-4975-acd9-c94808417877 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:27.103859 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76ea3f-3bdd-4975-acd9-c94808417877 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:27.103859 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76ea3f-3bdd-4975-acd9-c94808417877 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:27.105901 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 581/1154] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:15:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:15:27.114810 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32093786-abed-4446-a714-3f2d6e0bf37f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:27.115252 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32093786-abed-4446-a714-3f2d6e0bf37f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:27.115400 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32093786-abed-4446-a714-3f2d6e0bf37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:27.140963 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-32093786-abed-4446-a714-3f2d6e0bf37f None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:15:27.142145 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-32093786-abed-4446-a714-3f2d6e0bf37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:15:27.196072 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:27.217078 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 574/1155] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:15:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:15:27.243436 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a76ea3f-3bdd-4975-acd9-c94808417877 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:27.354126 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a76ea3f-3bdd-4975-acd9-c94808417877 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:27.388653 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d82902-0c65-44fa-adb8-a8bc5a4cdf7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:27.389275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d82902-0c65-44fa-adb8-a8bc5a4cdf7c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:27.389550 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d82902-0c65-44fa-adb8-a8bc5a4cdf7c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:27.389815 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d82902-0c65-44fa-adb8-a8bc5a4cdf7c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:27.401355 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d82902-0c65-44fa-adb8-a8bc5a4cdf7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:15:27.406155 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-94d82902-0c65-44fa-adb8-a8bc5a4cdf7c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:15:27.406155 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:15:27.406155 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:15:27.406155 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:15:27.406155 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:15:27.406155 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:15:27.406155 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:15:27.406155 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:15:27.406155 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:15:27.406155 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:15:27.406155 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:15:27.406155 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:15:27.406155 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:15:27.406155 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:15:27.406155 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:15:27.406155 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:15:27.407266 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:15:27.407266 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:15:27.407266 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:15:27.407266 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:15:27.407266 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:15:27.407266 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:15:27.407266 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:15:27.407266 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:15:27.407266 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:15:27.407266 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:15:27.407266 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:15:27.407266 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:15:27.407266 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:15:27.407266 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:15:27.407266 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:15:27.407266 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:15:27.408287 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:15:27.408287 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:15:27.408287 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:15:27.408287 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:15:27.408287 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:15:27.408287 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:15:27.408287 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:15:27.408287 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:15:27.408287 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:15:27.408287 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:15:27.408287 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:15:27.408287 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:15:27.411072 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 582/1156] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 10:15:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:15:27.413634 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32093786-abed-4446-a714-3f2d6e0bf37f None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:27.518457 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32093786-abed-4446-a714-3f2d6e0bf37f None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:27.558275 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90c574-f7fb-46fc-a321-31ccd12562f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:27.558882 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90c574-f7fb-46fc-a321-31ccd12562f6 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:27.559190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90c574-f7fb-46fc-a321-31ccd12562f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:27.559454 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90c574-f7fb-46fc-a321-31ccd12562f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:27.560162 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a90c574-f7fb-46fc-a321-31ccd12562f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:15:27.560461 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a90c574-f7fb-46fc-a321-31ccd12562f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:27.561850 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a90c574-f7fb-46fc-a321-31ccd12562f6 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:27.569963 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 575/1157] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:15:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:27.578936 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d82902-0c65-44fa-adb8-a8bc5a4cdf7c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:27.634552 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:27.693182 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d82902-0c65-44fa-adb8-a8bc5a4cdf7c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:27.729999 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13ce37-2253-414d-b9e5-f96089e3151b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:27.730684 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13ce37-2253-414d-b9e5-f96089e3151b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:27.730963 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13ce37-2253-414d-b9e5-f96089e3151b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:27.731264 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13ce37-2253-414d-b9e5-f96089e3151b None admin] PATH_INFO: `/v3/users/a983d4a108c04e94aa9666f614e0f4fe` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:27.732540 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd13ce37-2253-414d-b9e5-f96089e3151b None admin] RBAC: Authorizing `identity:get_user(user_id=a983d4a108c04e94aa9666f614e0f4fe)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:27.733971 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd13ce37-2253-414d-b9e5-f96089e3151b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:27.735598 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 583/1158] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:15:27 2022] GET /identity/v3/users/a983d4a108c04e94aa9666f614e0f4fe => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:27.744500 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90c574-f7fb-46fc-a321-31ccd12562f6 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:27.799725 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:27.861470 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90c574-f7fb-46fc-a321-31ccd12562f6 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:27.895859 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb81665-1fa8-412e-92a1-234a7a2d5aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:27.896405 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb81665-1fa8-412e-92a1-234a7a2d5aad None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:27.896687 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb81665-1fa8-412e-92a1-234a7a2d5aad None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:27.896965 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb81665-1fa8-412e-92a1-234a7a2d5aad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:27.967104 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb81665-1fa8-412e-92a1-234a7a2d5aad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:27.968571 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb81665-1fa8-412e-92a1-234a7a2d5aad None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:28.014028 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:28.073193 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 576/1159] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:15:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 10:15:28.079453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13ce37-2253-414d-b9e5-f96089e3151b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:28.195935 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13ce37-2253-414d-b9e5-f96089e3151b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:28.208783 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:28.235561 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3512f3a9-cffb-438b-b180-d0df788e88c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:28.236518 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512f3a9-cffb-438b-b180-d0df788e88c9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:28.236907 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512f3a9-cffb-438b-b180-d0df788e88c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:28.237277 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512f3a9-cffb-438b-b180-d0df788e88c9 None admin] PATH_INFO: `/v3/projects/316cd09f851e41ae8252ddc4c39fc803` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:28.239275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3512f3a9-cffb-438b-b180-d0df788e88c9 None admin] RBAC: Authorizing `identity:get_project(project_id=316cd09f851e41ae8252ddc4c39fc803)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:28.241925 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3512f3a9-cffb-438b-b180-d0df788e88c9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:28.243655 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 584/1160] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:15:28 2022] GET /identity/v3/projects/316cd09f851e41ae8252ddc4c39fc803 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:28.250159 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb81665-1fa8-412e-92a1-234a7a2d5aad None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:28.349755 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb81665-1fa8-412e-92a1-234a7a2d5aad None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:28.384783 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb0e23-1c46-43e8-86b5-9bc9eb7b547c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:28.385433 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb0e23-1c46-43e8-86b5-9bc9eb7b547c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:28.385710 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb0e23-1c46-43e8-86b5-9bc9eb7b547c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:28.385975 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb0e23-1c46-43e8-86b5-9bc9eb7b547c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:28.386755 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb0e23-1c46-43e8-86b5-9bc9eb7b547c None admin] RBAC: Adding query filter params (role.id=daef89adcaeb48a382a0bcb03340ccbb, scope.project.id=316cd09f851e41ae8252ddc4c39fc803, user.id=a983d4a108c04e94aa9666f614e0f4fe) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:15:28.387094 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb0e23-1c46-43e8-86b5-9bc9eb7b547c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:28.388513 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb0e23-1c46-43e8-86b5-9bc9eb7b547c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:28.394326 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 577/1161] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:15:28 2022] GET /identity/v3/role_assignments?user.id=a983d4a108c04e94aa9666f614e0f4fe&scope.project.id=316cd09f851e41ae8252ddc4c39fc803&role.id=daef89adcaeb48a382a0bcb03340ccbb => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:30.398981 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c99620-8ee5-4c9b-b055-2a1c25b2774d None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:30.399882 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c99620-8ee5-4c9b-b055-2a1c25b2774d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:30.399882 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c99620-8ee5-4c9b-b055-2a1c25b2774d None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:30.402081 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 585/1162] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:15:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:15:30.408405 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b3fa14-a0cd-4be5-a641-1e42fd7f6357 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:30.408819 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b3fa14-a0cd-4be5-a641-1e42fd7f6357 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:30.408924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b3fa14-a0cd-4be5-a641-1e42fd7f6357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:30.436038 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-57b3fa14-a0cd-4be5-a641-1e42fd7f6357 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:15:30.437218 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-57b3fa14-a0cd-4be5-a641-1e42fd7f6357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:15:30.440029 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:30.506924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 578/1163] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:15:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:15:30.529072 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c99620-8ee5-4c9b-b055-2a1c25b2774d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:30.637009 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c99620-8ee5-4c9b-b055-2a1c25b2774d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:30.671292 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8078e4-430b-4d5e-a67c-fcea8f475fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:30.671832 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8078e4-430b-4d5e-a67c-fcea8f475fa4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:30.672108 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8078e4-430b-4d5e-a67c-fcea8f475fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:30.672369 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8078e4-430b-4d5e-a67c-fcea8f475fa4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:30.720457 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:30.740815 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8078e4-430b-4d5e-a67c-fcea8f475fa4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:30.742424 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8078e4-430b-4d5e-a67c-fcea8f475fa4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:30.851908 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 586/1164] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:15:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 10:15:30.860941 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b3fa14-a0cd-4be5-a641-1e42fd7f6357 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:30.966103 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b3fa14-a0cd-4be5-a641-1e42fd7f6357 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:31.008245 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7764b4-f473-43f2-b524-4217ca646847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:31.008880 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7764b4-f473-43f2-b524-4217ca646847 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:31.009154 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7764b4-f473-43f2-b524-4217ca646847 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:31.009422 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7764b4-f473-43f2-b524-4217ca646847 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:31.016548 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7764b4-f473-43f2-b524-4217ca646847 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:31.017979 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7764b4-f473-43f2-b524-4217ca646847 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:31.020817 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:31.031277 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-ed7764b4-f473-43f2-b524-4217ca646847 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 05 10:15:31.031277 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:15:31.031277 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 05 10:15:31.031277 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 05 10:15:31.031277 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 10:15:31.031277 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 10:15:31.031277 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 10:15:31.031277 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 10:15:31.031277 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 05 10:15:31.031277 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:15:31.031277 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 10:15:31.031277 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:15:31.031277 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:15:31.031277 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:15:31.031277 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:15:31.031277 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:15:31.031277 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:15:31.032532 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:15:31.032532 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:15:31.032532 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:15:31.032532 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:15:31.032532 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:15:31.032532 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:15:31.032532 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:15:31.032532 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:15:31.032532 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:15:31.032532 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:15:31.032532 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:15:31.032532 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:15:31.032532 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:15:31.032532 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:15:31.032532 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:15:31.032532 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:15:31.034184 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:15:31.034184 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:15:31.034184 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:15:31.034184 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:15:31.034184 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:15:31.034184 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:15:31.034184 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:15:31.034184 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:15:31.034184 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:15:31.034184 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:15:31.034184 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 05 10:15:31.034184 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:15:31.034184 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 05 10:15:31.034184 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:15:31.034184 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 579/1165] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Aug 5 10:15:30 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:15:31.040854 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8078e4-430b-4d5e-a67c-fcea8f475fa4 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:31.124415 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:31.144341 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8078e4-430b-4d5e-a67c-fcea8f475fa4 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:31.180262 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4632fe7d-d75a-4192-ad40-b342d819e767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:31.180905 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632fe7d-d75a-4192-ad40-b342d819e767 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:31.181163 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632fe7d-d75a-4192-ad40-b342d819e767 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:31.181433 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632fe7d-d75a-4192-ad40-b342d819e767 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:31.182119 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4632fe7d-d75a-4192-ad40-b342d819e767 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:15:31.182430 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4632fe7d-d75a-4192-ad40-b342d819e767 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:31.183826 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4632fe7d-d75a-4192-ad40-b342d819e767 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:31.191078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 587/1166] 10.222.0.40 () {56 vars in 1115 bytes} [Fri Aug 5 10:15:31 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:33.191133 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc8a44-e4f6-4914-a5e5-3ccfd58263d7 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:33.191651 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc8a44-e4f6-4914-a5e5-3ccfd58263d7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:33.191651 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc8a44-e4f6-4914-a5e5-3ccfd58263d7 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:33.193059 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 580/1167] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:15:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:15:33.199723 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4f838-bf12-485f-82e6-6d363c8dc798 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:33.200152 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4f838-bf12-485f-82e6-6d363c8dc798 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:33.200291 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4f838-bf12-485f-82e6-6d363c8dc798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:33.228847 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-96c4f838-bf12-485f-82e6-6d363c8dc798 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:15:33.230046 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-96c4f838-bf12-485f-82e6-6d363c8dc798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:15:33.301215 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 588/1168] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:15:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:15:33.324699 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffc8a44-e4f6-4914-a5e5-3ccfd58263d7 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:33.438677 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffc8a44-e4f6-4914-a5e5-3ccfd58263d7 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:33.473744 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3574456-e051-4fb6-88b6-6b758ab87f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:33.474351 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3574456-e051-4fb6-88b6-6b758ab87f5f None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:33.474632 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3574456-e051-4fb6-88b6-6b758ab87f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:33.475106 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3574456-e051-4fb6-88b6-6b758ab87f5f None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:33.475824 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3574456-e051-4fb6-88b6-6b758ab87f5f None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:33.477204 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3574456-e051-4fb6-88b6-6b758ab87f5f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:33.497368 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 581/1169] 10.222.0.40 () {60 vars in 1138 bytes} [Fri Aug 5 10:15:33 2022] POST /identity/v3/domains => generated 267 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:15:35.512829 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfaf690-7cc4-4050-ad8e-c1398cc04836 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:35.513681 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfaf690-7cc4-4050-ad8e-c1398cc04836 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:35.513681 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfaf690-7cc4-4050-ad8e-c1398cc04836 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:35.515864 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 589/1170] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:15:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:15:35.523758 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa298b0b-be34-4504-b304-50691feb9e49 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:35.524219 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa298b0b-be34-4504-b304-50691feb9e49 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:35.524310 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa298b0b-be34-4504-b304-50691feb9e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:35.544317 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-aa298b0b-be34-4504-b304-50691feb9e49 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:15:35.545513 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-aa298b0b-be34-4504-b304-50691feb9e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:15:35.605863 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:35.622271 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 582/1171] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:15:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 10:15:35.645468 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfaf690-7cc4-4050-ad8e-c1398cc04836 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:35.731903 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:35.760613 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfaf690-7cc4-4050-ad8e-c1398cc04836 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:35.798576 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb8f50a-e152-4563-8e37-7b0261d00de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:35.799166 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb8f50a-e152-4563-8e37-7b0261d00de6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:35.799443 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb8f50a-e152-4563-8e37-7b0261d00de6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:35.799714 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb8f50a-e152-4563-8e37-7b0261d00de6 None admin] PATH_INFO: `/v3/domains/d473ed4c8eb5407a93e816ad7d30a041` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:35.810680 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb8f50a-e152-4563-8e37-7b0261d00de6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d473ed4c8eb5407a93e816ad7d30a041)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:35.812166 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb8f50a-e152-4563-8e37-7b0261d00de6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:35.813765 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 590/1172] 10.222.0.40 () {56 vars in 1183 bytes} [Fri Aug 5 10:15:35 2022] GET /identity/v3/domains/d473ed4c8eb5407a93e816ad7d30a041 => generated 267 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:35.823982 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa298b0b-be34-4504-b304-50691feb9e49 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:35.934732 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa298b0b-be34-4504-b304-50691feb9e49 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:35.970332 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1f3d0d-65ff-4393-a1a0-f77a365680ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:35.970943 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f3d0d-65ff-4393-a1a0-f77a365680ba None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:35.971249 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f3d0d-65ff-4393-a1a0-f77a365680ba None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:35.971520 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f3d0d-65ff-4393-a1a0-f77a365680ba None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:35.972193 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1f3d0d-65ff-4393-a1a0-f77a365680ba None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:35.973625 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1f3d0d-65ff-4393-a1a0-f77a365680ba None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:36.003920 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 583/1173] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 5 10:15:35 2022] POST /identity/v3/projects => generated 361 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:15:38.066362 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9e550-bcc9-4fd5-a067-061c2b97bf3e None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:38.067269 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9e550-bcc9-4fd5-a067-061c2b97bf3e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:38.067269 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9e550-bcc9-4fd5-a067-061c2b97bf3e None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:38.069572 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 591/1174] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:15:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:15:38.074866 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f854a0-ff9a-401c-9275-23ef511310ef None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:38.075320 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f854a0-ff9a-401c-9275-23ef511310ef None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:38.075411 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f854a0-ff9a-401c-9275-23ef511310ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:38.094591 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-92f854a0-ff9a-401c-9275-23ef511310ef None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:15:38.095795 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-92f854a0-ff9a-401c-9275-23ef511310ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:15:38.131911 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:38.168346 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 584/1175] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:15:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:15:38.193159 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9e550-bcc9-4fd5-a067-061c2b97bf3e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:38.325217 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9e550-bcc9-4fd5-a067-061c2b97bf3e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:38.365375 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364440c6-18cc-4b95-a945-2ec7538526f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:38.366009 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364440c6-18cc-4b95-a945-2ec7538526f2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:38.366282 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364440c6-18cc-4b95-a945-2ec7538526f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:38.366549 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364440c6-18cc-4b95-a945-2ec7538526f2 None admin] PATH_INFO: `/v3/domains/d473ed4c8eb5407a93e816ad7d30a041` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:38.367847 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364440c6-18cc-4b95-a945-2ec7538526f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d473ed4c8eb5407a93e816ad7d30a041)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:38.369302 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364440c6-18cc-4b95-a945-2ec7538526f2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:38.370891 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 592/1176] 10.222.0.40 () {56 vars in 1183 bytes} [Fri Aug 5 10:15:38 2022] GET /identity/v3/domains/d473ed4c8eb5407a93e816ad7d30a041 => generated 267 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:38.376946 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f854a0-ff9a-401c-9275-23ef511310ef None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:38.478576 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f854a0-ff9a-401c-9275-23ef511310ef None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:38.513126 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74797365-170f-40fb-917c-e4da543dfc48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:38.513722 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74797365-170f-40fb-917c-e4da543dfc48 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:38.514000 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74797365-170f-40fb-917c-e4da543dfc48 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:38.514268 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74797365-170f-40fb-917c-e4da543dfc48 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:38.514950 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74797365-170f-40fb-917c-e4da543dfc48 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:38.516389 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74797365-170f-40fb-917c-e4da543dfc48 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:38.540255 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 585/1177] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 5 10:15:38 2022] POST /identity/v3/users => generated 311 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:15:40.587435 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9f289-2ff4-4366-b337-0f5a15e64848 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:40.588281 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9f289-2ff4-4366-b337-0f5a15e64848 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:40.588281 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9f289-2ff4-4366-b337-0f5a15e64848 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:40.590525 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 593/1178] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:15:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:15:40.596925 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2c7c2-ee7a-42aa-b473-0e70e3595a87 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:40.597609 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2c7c2-ee7a-42aa-b473-0e70e3595a87 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:40.597749 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2c7c2-ee7a-42aa-b473-0e70e3595a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:40.619915 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-83f2c7c2-ee7a-42aa-b473-0e70e3595a87 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:15:40.621085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-83f2c7c2-ee7a-42aa-b473-0e70e3595a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:15:40.646806 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:40.700905 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 586/1179] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:15:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:15:40.730948 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c9f289-2ff4-4366-b337-0f5a15e64848 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:40.735579 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:40.845799 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c9f289-2ff4-4366-b337-0f5a15e64848 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:40.882272 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a5bbb-b337-4095-ab4d-86138cf14232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:40.882948 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a5bbb-b337-4095-ab4d-86138cf14232 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:40.883222 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a5bbb-b337-4095-ab4d-86138cf14232 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:40.883501 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a5bbb-b337-4095-ab4d-86138cf14232 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:40.896372 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a5bbb-b337-4095-ab4d-86138cf14232 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:15:40.901267 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-c65a5bbb-b337-4095-ab4d-86138cf14232 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:15:40.901267 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:15:40.901267 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:15:40.901267 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:15:40.901267 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:15:40.901267 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:15:40.901267 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:15:40.901267 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:15:40.901267 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:15:40.901267 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:15:40.901267 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:15:40.901267 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:15:40.901267 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:15:40.901267 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:15:40.901267 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:15:40.901267 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:15:40.902367 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:15:40.902367 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:15:40.902367 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:15:40.902367 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:15:40.902367 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:15:40.902367 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:15:40.902367 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:15:40.902367 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:15:40.902367 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:15:40.902367 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:15:40.902367 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:15:40.902367 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:15:40.902367 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:15:40.902367 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:15:40.902367 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:15:40.902367 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:15:40.908042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:15:40.908042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:15:40.908042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:15:40.908042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:15:40.908042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:15:40.908042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:15:40.908042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:15:40.908042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:15:40.908042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:15:40.908042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:15:40.908042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:15:40.908042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:15:40.908042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 594/1180] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 10:15:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:15:40.910243 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2c7c2-ee7a-42aa-b473-0e70e3595a87 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:41.014244 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2c7c2-ee7a-42aa-b473-0e70e3595a87 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:41.050888 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9fcbe0-373a-4cf8-82e9-a916d3f464ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:41.051543 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9fcbe0-373a-4cf8-82e9-a916d3f464ca None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:41.051827 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9fcbe0-373a-4cf8-82e9-a916d3f464ca None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:41.052099 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9fcbe0-373a-4cf8-82e9-a916d3f464ca None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:41.052821 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9fcbe0-373a-4cf8-82e9-a916d3f464ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:15:41.053123 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9fcbe0-373a-4cf8-82e9-a916d3f464ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:41.054605 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9fcbe0-373a-4cf8-82e9-a916d3f464ca None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:41.062924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 587/1181] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:15:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:41.071707 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a5bbb-b337-4095-ab4d-86138cf14232 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:41.143961 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:41.187031 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a5bbb-b337-4095-ab4d-86138cf14232 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:41.219954 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b852084b-86b0-43ff-8b3e-d68c4b96c2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:41.220591 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852084b-86b0-43ff-8b3e-d68c4b96c2a7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:41.220860 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852084b-86b0-43ff-8b3e-d68c4b96c2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:41.221130 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852084b-86b0-43ff-8b3e-d68c4b96c2a7 None admin] PATH_INFO: `/v3/users/5fdccf7b8eef4aee94c4a218f790c8fc` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:41.234579 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b852084b-86b0-43ff-8b3e-d68c4b96c2a7 None admin] RBAC: Authorizing `identity:get_user(user_id=5fdccf7b8eef4aee94c4a218f790c8fc)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:41.236053 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b852084b-86b0-43ff-8b3e-d68c4b96c2a7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:41.237667 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 595/1182] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:15:41 2022] GET /identity/v3/users/5fdccf7b8eef4aee94c4a218f790c8fc => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:41.244550 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9fcbe0-373a-4cf8-82e9-a916d3f464ca None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:41.272147 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:41.356630 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9fcbe0-373a-4cf8-82e9-a916d3f464ca None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:41.390604 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1604c5b-61bd-45f6-b582-85a4631cff00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:41.391256 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1604c5b-61bd-45f6-b582-85a4631cff00 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:41.391537 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1604c5b-61bd-45f6-b582-85a4631cff00 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:41.391807 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1604c5b-61bd-45f6-b582-85a4631cff00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:41.462302 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1604c5b-61bd-45f6-b582-85a4631cff00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:41.463828 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1604c5b-61bd-45f6-b582-85a4631cff00 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:41.484622 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:41.578524 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 588/1183] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:15:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 10:15:41.587297 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b852084b-86b0-43ff-8b3e-d68c4b96c2a7 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:41.688280 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b852084b-86b0-43ff-8b3e-d68c4b96c2a7 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:41.708077 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:41.732064 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062ebab-af48-4d5f-8b5b-547bbdcc6f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:41.732956 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062ebab-af48-4d5f-8b5b-547bbdcc6f3d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:41.733413 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062ebab-af48-4d5f-8b5b-547bbdcc6f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:41.733854 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062ebab-af48-4d5f-8b5b-547bbdcc6f3d None admin] PATH_INFO: `/v3/projects/12aadc0076844c7c88061ebb46e6a260` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:41.735773 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f062ebab-af48-4d5f-8b5b-547bbdcc6f3d None admin] RBAC: Authorizing `identity:get_project(project_id=12aadc0076844c7c88061ebb46e6a260)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:41.738080 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f062ebab-af48-4d5f-8b5b-547bbdcc6f3d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:41.740824 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 596/1184] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:15:41 2022] GET /identity/v3/projects/12aadc0076844c7c88061ebb46e6a260 => generated 361 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:41.747220 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1604c5b-61bd-45f6-b582-85a4631cff00 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:41.846359 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1604c5b-61bd-45f6-b582-85a4631cff00 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:41.866710 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:41.888725 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5edc862-82a7-4348-9234-4d4c75588cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:41.889580 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5edc862-82a7-4348-9234-4d4c75588cbe None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:41.889939 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5edc862-82a7-4348-9234-4d4c75588cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:41.890270 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5edc862-82a7-4348-9234-4d4c75588cbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:41.891512 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5edc862-82a7-4348-9234-4d4c75588cbe None admin] RBAC: Adding query filter params (role.id=daef89adcaeb48a382a0bcb03340ccbb, scope.project.id=12aadc0076844c7c88061ebb46e6a260, user.id=5fdccf7b8eef4aee94c4a218f790c8fc) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:15:41.891922 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5edc862-82a7-4348-9234-4d4c75588cbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:41.894126 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5edc862-82a7-4348-9234-4d4c75588cbe None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:41.901182 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 589/1185] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:15:41 2022] GET /identity/v3/role_assignments?user.id=5fdccf7b8eef4aee94c4a218f790c8fc&scope.project.id=12aadc0076844c7c88061ebb46e6a260&role.id=daef89adcaeb48a382a0bcb03340ccbb => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:43.873844 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f48e71-b7e7-49f7-8774-b41c1c12e1c5 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:43.874865 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f48e71-b7e7-49f7-8774-b41c1c12e1c5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:43.874865 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f48e71-b7e7-49f7-8774-b41c1c12e1c5 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:43.877009 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 597/1186] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:15:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:15:43.882890 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5b943-e0b6-4990-af8c-ef0e6f4d1840 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:43.883356 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5b943-e0b6-4990-af8c-ef0e6f4d1840 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:43.883452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5b943-e0b6-4990-af8c-ef0e6f4d1840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:43.903276 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-54d5b943-e0b6-4990-af8c-ef0e6f4d1840 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:15:43.904433 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-54d5b943-e0b6-4990-af8c-ef0e6f4d1840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:15:43.975821 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 590/1187] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:15:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:15:44.004813 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f48e71-b7e7-49f7-8774-b41c1c12e1c5 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:44.119615 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f48e71-b7e7-49f7-8774-b41c1c12e1c5 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:44.155361 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ba723c-2534-4d5d-ae13-2704979b1c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:44.155929 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba723c-2534-4d5d-ae13-2704979b1c77 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:44.156357 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba723c-2534-4d5d-ae13-2704979b1c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:44.156627 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba723c-2534-4d5d-ae13-2704979b1c77 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:44.171331 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ba723c-2534-4d5d-ae13-2704979b1c77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:15:44.176165 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-60ba723c-2534-4d5d-ae13-2704979b1c77 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:15:44.176165 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:15:44.176165 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:15:44.176165 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:15:44.176165 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:15:44.176165 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:15:44.176165 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:15:44.176165 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:15:44.176165 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:15:44.176165 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:15:44.176165 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:15:44.176165 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:15:44.176165 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:15:44.176165 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:15:44.176165 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:15:44.176165 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:15:44.177320 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:15:44.177320 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:15:44.177320 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:15:44.177320 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:15:44.177320 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:15:44.177320 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:15:44.177320 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:15:44.177320 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:15:44.177320 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:15:44.177320 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:15:44.177320 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:15:44.177320 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:15:44.177320 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:15:44.177320 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:15:44.177320 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:15:44.177320 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:15:44.178664 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:15:44.178664 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:15:44.178664 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:15:44.178664 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:15:44.178664 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:15:44.178664 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:15:44.178664 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:15:44.178664 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:15:44.178664 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:15:44.178664 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:15:44.178664 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:15:44.178664 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:15:44.178664 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 598/1188] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 10:15:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:15:44.183643 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d5b943-e0b6-4990-af8c-ef0e6f4d1840 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:44.286919 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d5b943-e0b6-4990-af8c-ef0e6f4d1840 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:44.301734 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:44.330492 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0caf8bc-297c-41d2-bdaf-7c6b2fbb871e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:44.331138 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0caf8bc-297c-41d2-bdaf-7c6b2fbb871e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:44.331420 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0caf8bc-297c-41d2-bdaf-7c6b2fbb871e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:44.331692 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0caf8bc-297c-41d2-bdaf-7c6b2fbb871e None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:44.332410 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0caf8bc-297c-41d2-bdaf-7c6b2fbb871e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:15:44.332714 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0caf8bc-297c-41d2-bdaf-7c6b2fbb871e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:44.334090 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0caf8bc-297c-41d2-bdaf-7c6b2fbb871e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:44.342716 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 591/1189] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:15:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:44.348783 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ba723c-2534-4d5d-ae13-2704979b1c77 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:44.400154 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:44.466418 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ba723c-2534-4d5d-ae13-2704979b1c77 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:44.501473 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321e6bf2-c563-45d0-ba4b-298dd5128bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:44.502095 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e6bf2-c563-45d0-ba4b-298dd5128bd6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:44.502371 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e6bf2-c563-45d0-ba4b-298dd5128bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:44.502636 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e6bf2-c563-45d0-ba4b-298dd5128bd6 None admin] PATH_INFO: `/v3/users/5fdccf7b8eef4aee94c4a218f790c8fc` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:44.503862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321e6bf2-c563-45d0-ba4b-298dd5128bd6 None admin] RBAC: Authorizing `identity:get_user(user_id=5fdccf7b8eef4aee94c4a218f790c8fc)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:44.505296 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321e6bf2-c563-45d0-ba4b-298dd5128bd6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:44.506884 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 599/1190] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:15:44 2022] GET /identity/v3/users/5fdccf7b8eef4aee94c4a218f790c8fc => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:44.515505 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0caf8bc-297c-41d2-bdaf-7c6b2fbb871e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:44.636468 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0caf8bc-297c-41d2-bdaf-7c6b2fbb871e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:44.683831 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6cc06b-d781-434c-bfb4-b00c3f54a33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:44.684462 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6cc06b-d781-434c-bfb4-b00c3f54a33b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:44.684740 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6cc06b-d781-434c-bfb4-b00c3f54a33b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:44.685003 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6cc06b-d781-434c-bfb4-b00c3f54a33b None admin] PATH_INFO: `/v3/projects/12aadc0076844c7c88061ebb46e6a260` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:44.686279 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6cc06b-d781-434c-bfb4-b00c3f54a33b None admin] RBAC: Authorizing `identity:get_project(project_id=12aadc0076844c7c88061ebb46e6a260)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:44.687743 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6cc06b-d781-434c-bfb4-b00c3f54a33b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:44.689400 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 592/1191] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:15:44 2022] GET /identity/v3/projects/12aadc0076844c7c88061ebb46e6a260 => generated 361 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:44.698226 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321e6bf2-c563-45d0-ba4b-298dd5128bd6 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:44.801644 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321e6bf2-c563-45d0-ba4b-298dd5128bd6 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:44.819059 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:44.838663 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f344486-73be-4009-9791-6463c39e4125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:44.839362 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f344486-73be-4009-9791-6463c39e4125 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:44.839640 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f344486-73be-4009-9791-6463c39e4125 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:44.839914 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f344486-73be-4009-9791-6463c39e4125 None admin] PATH_INFO: `/v3/projects/12aadc0076844c7c88061ebb46e6a260/users/5fdccf7b8eef4aee94c4a218f790c8fc/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:44.842128 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f344486-73be-4009-9791-6463c39e4125 None admin] RBAC: Authorizing `identity:create_grant(project_id=12aadc0076844c7c88061ebb46e6a260, user_id=5fdccf7b8eef4aee94c4a218f790c8fc, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:44.843590 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f344486-73be-4009-9791-6463c39e4125 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:44.859928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 600/1192] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 10:15:44 2022] PUT /identity/v3/projects/12aadc0076844c7c88061ebb46e6a260/users/5fdccf7b8eef4aee94c4a218f790c8fc/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:15:46.811718 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a57c6-f1de-4d8b-b60d-19ca65f36b32 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:46.812330 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a57c6-f1de-4d8b-b60d-19ca65f36b32 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:46.812330 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a57c6-f1de-4d8b-b60d-19ca65f36b32 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:46.813639 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 593/1193] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:15:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:15:46.820034 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267848e7-4126-4a00-974e-60825174d6c3 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:46.820861 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267848e7-4126-4a00-974e-60825174d6c3 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:46.821027 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267848e7-4126-4a00-974e-60825174d6c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:46.845933 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-267848e7-4126-4a00-974e-60825174d6c3 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:15:46.847170 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-267848e7-4126-4a00-974e-60825174d6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:15:46.920589 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 601/1194] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:15:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:15:46.944011 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940a57c6-f1de-4d8b-b60d-19ca65f36b32 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:46.995506 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:47.063048 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940a57c6-f1de-4d8b-b60d-19ca65f36b32 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:47.106305 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3fabce-4b97-48a0-b4b0-1ea83dbbfaf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:47.106943 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3fabce-4b97-48a0-b4b0-1ea83dbbfaf6 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:47.107233 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3fabce-4b97-48a0-b4b0-1ea83dbbfaf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:47.107496 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3fabce-4b97-48a0-b4b0-1ea83dbbfaf6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:47.118393 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3fabce-4b97-48a0-b4b0-1ea83dbbfaf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:15:47.123234 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-4e3fabce-4b97-48a0-b4b0-1ea83dbbfaf6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:15:47.123234 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:15:47.123234 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:15:47.123234 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:15:47.123234 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:15:47.123234 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:15:47.123234 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:15:47.123234 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:15:47.123234 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:15:47.123234 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:15:47.123234 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:15:47.123234 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:15:47.123234 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:15:47.123234 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:15:47.123234 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:15:47.123234 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:15:47.124338 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:15:47.124338 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:15:47.124338 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:15:47.124338 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:15:47.124338 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:15:47.124338 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:15:47.124338 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:15:47.124338 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:15:47.124338 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:15:47.124338 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:15:47.124338 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:15:47.124338 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:15:47.124338 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:15:47.124338 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:15:47.124338 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:15:47.124338 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:15:47.125527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:15:47.125527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:15:47.125527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:15:47.125527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:15:47.125527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:15:47.125527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:15:47.125527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:15:47.125527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:15:47.125527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:15:47.125527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:15:47.125527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:15:47.125527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:15:47.125527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 594/1195] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 10:15:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:15:47.132287 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267848e7-4126-4a00-974e-60825174d6c3 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:47.243254 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267848e7-4126-4a00-974e-60825174d6c3 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:47.250263 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:47.280921 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39de65c5-ada8-4608-8ed9-42727b30e0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:47.281543 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de65c5-ada8-4608-8ed9-42727b30e0b7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:47.281814 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de65c5-ada8-4608-8ed9-42727b30e0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:47.282083 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de65c5-ada8-4608-8ed9-42727b30e0b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:47.282787 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39de65c5-ada8-4608-8ed9-42727b30e0b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:15:47.283112 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39de65c5-ada8-4608-8ed9-42727b30e0b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:47.284499 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39de65c5-ada8-4608-8ed9-42727b30e0b7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:47.293585 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 602/1196] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:15:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:47.299853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3fabce-4b97-48a0-b4b0-1ea83dbbfaf6 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:47.419047 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3fabce-4b97-48a0-b4b0-1ea83dbbfaf6 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:47.421239 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:47.454644 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71745888-d257-4434-aade-68fad93c87bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:47.455604 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71745888-d257-4434-aade-68fad93c87bf None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:47.456022 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71745888-d257-4434-aade-68fad93c87bf None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:47.456408 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71745888-d257-4434-aade-68fad93c87bf None admin] PATH_INFO: `/v3/users/5fdccf7b8eef4aee94c4a218f790c8fc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:47.458488 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71745888-d257-4434-aade-68fad93c87bf None admin] RBAC: Authorizing `identity:get_user(user_id=5fdccf7b8eef4aee94c4a218f790c8fc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:47.460836 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71745888-d257-4434-aade-68fad93c87bf None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:47.463753 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 595/1197] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 10:15:47 2022] GET /identity/v3/users/5fdccf7b8eef4aee94c4a218f790c8fc => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:47.472601 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39de65c5-ada8-4608-8ed9-42727b30e0b7 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:47.586078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39de65c5-ada8-4608-8ed9-42727b30e0b7 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:47.621768 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396a1e0-b5b0-4841-b4a4-6d53c7004be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:47.622374 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396a1e0-b5b0-4841-b4a4-6d53c7004be9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:47.622648 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396a1e0-b5b0-4841-b4a4-6d53c7004be9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:47.622944 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396a1e0-b5b0-4841-b4a4-6d53c7004be9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:47.656714 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:47.698580 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5396a1e0-b5b0-4841-b4a4-6d53c7004be9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:47.700002 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5396a1e0-b5b0-4841-b4a4-6d53c7004be9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:47.800848 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 603/1198] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:15:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 10:15:47.807129 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71745888-d257-4434-aade-68fad93c87bf None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:47.912966 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71745888-d257-4434-aade-68fad93c87bf None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:47.948204 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c90c0-68d8-444f-8c79-5042d97f9054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:47.948906 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c90c0-68d8-444f-8c79-5042d97f9054 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:47.949122 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c90c0-68d8-444f-8c79-5042d97f9054 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:47.949388 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c90c0-68d8-444f-8c79-5042d97f9054 None admin] PATH_INFO: `/v3/projects/12aadc0076844c7c88061ebb46e6a260` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:47.950662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7c90c0-68d8-444f-8c79-5042d97f9054 None admin] RBAC: Authorizing `identity:get_project(project_id=12aadc0076844c7c88061ebb46e6a260)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:47.952139 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7c90c0-68d8-444f-8c79-5042d97f9054 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:47.953805 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 596/1199] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 10:15:47 2022] GET /identity/v3/projects/12aadc0076844c7c88061ebb46e6a260 => generated 361 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:47.960299 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396a1e0-b5b0-4841-b4a4-6d53c7004be9 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:15:48.025246 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:48.063281 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396a1e0-b5b0-4841-b4a4-6d53c7004be9 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:15:48.098461 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f349df-afe0-4fd1-8ce6-7e30120c2560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:15:48.099094 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f349df-afe0-4fd1-8ce6-7e30120c2560 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:15:48.099369 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f349df-afe0-4fd1-8ce6-7e30120c2560 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:15:48.099638 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f349df-afe0-4fd1-8ce6-7e30120c2560 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:15:48.100419 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f349df-afe0-4fd1-8ce6-7e30120c2560 None admin] RBAC: Adding query filter params (role.id=daef89adcaeb48a382a0bcb03340ccbb, scope.project.id=12aadc0076844c7c88061ebb46e6a260, user.id=5fdccf7b8eef4aee94c4a218f790c8fc) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:15:48.100721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f349df-afe0-4fd1-8ce6-7e30120c2560 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:15:48.102224 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f349df-afe0-4fd1-8ce6-7e30120c2560 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:15:48.108089 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 604/1200] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:15:47 2022] GET /identity/v3/role_assignments?user.id=5fdccf7b8eef4aee94c4a218f790c8fc&scope.project.id=12aadc0076844c7c88061ebb46e6a260&role.id=daef89adcaeb48a382a0bcb03340ccbb => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:15:57.942764 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=80 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:15:58.106819 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=34 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:16:43.501343 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ae69a-0996-49fa-b907-6fb64cf2c0c7 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:43.501950 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ae69a-0996-49fa-b907-6fb64cf2c0c7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:43.501950 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ae69a-0996-49fa-b907-6fb64cf2c0c7 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:43.503657 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 597/1201] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:16:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:16:43.509173 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4982d6-0ff5-41b1-84d3-66ecfb746f63 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:43.509686 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4982d6-0ff5-41b1-84d3-66ecfb746f63 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:43.509776 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4982d6-0ff5-41b1-84d3-66ecfb746f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:43.532923 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-bc4982d6-0ff5-41b1-84d3-66ecfb746f63 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:16:43.534137 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bc4982d6-0ff5-41b1-84d3-66ecfb746f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:16:43.614072 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 605/1202] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:16:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:16:43.639494 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ae69a-0996-49fa-b907-6fb64cf2c0c7 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:16:43.761212 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ae69a-0996-49fa-b907-6fb64cf2c0c7 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:16:43.801607 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20466be6-59d8-4b8e-a678-688598c31d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:16:43.802610 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20466be6-59d8-4b8e-a678-688598c31d0b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:43.803119 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20466be6-59d8-4b8e-a678-688598c31d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:43.803377 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20466be6-59d8-4b8e-a678-688598c31d0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:43.817610 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20466be6-59d8-4b8e-a678-688598c31d0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:16:43.821897 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20466be6-59d8-4b8e-a678-688598c31d0b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:16:43.839865 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-20466be6-59d8-4b8e-a678-688598c31d0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:16:43.839865 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:16:43.839865 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:16:43.839865 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:16:43.839865 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:16:43.839865 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:16:43.839865 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:16:43.839865 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:16:43.839865 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:16:43.839865 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:16:43.839865 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:16:43.839865 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:16:43.839865 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:16:43.839865 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:16:43.839865 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:16:43.840946 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:16:43.840946 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:16:43.840946 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:16:43.840946 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:16:43.840946 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:16:43.840946 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:16:43.840946 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:16:43.840946 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:16:43.840946 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:16:43.840946 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:16:43.840946 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:16:43.840946 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:16:43.840946 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:16:43.840946 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:16:43.840946 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:16:43.840946 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:16:43.840946 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:16:43.841989 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:16:43.841989 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:16:43.841989 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:16:43.841989 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:16:43.841989 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 598/1203] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:16:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:16:43.849073 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4982d6-0ff5-41b1-84d3-66ecfb746f63 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:16:44.000639 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4982d6-0ff5-41b1-84d3-66ecfb746f63 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:16:44.042560 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:16:44.049406 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025826a8-0347-4206-9299-ecf4bfa4b780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:16:44.050072 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025826a8-0347-4206-9299-ecf4bfa4b780 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:44.050318 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025826a8-0347-4206-9299-ecf4bfa4b780 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:44.050690 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025826a8-0347-4206-9299-ecf4bfa4b780 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:44.051453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025826a8-0347-4206-9299-ecf4bfa4b780 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:16:44.051769 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025826a8-0347-4206-9299-ecf4bfa4b780 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:16:44.053116 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025826a8-0347-4206-9299-ecf4bfa4b780 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:16:44.065610 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 606/1204] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:16:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:16:44.071973 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20466be6-59d8-4b8e-a678-688598c31d0b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:16:44.142455 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:16:44.199266 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20466be6-59d8-4b8e-a678-688598c31d0b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:16:44.243252 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1569a80-47e9-45a8-8ba9-2d399719f292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:16:44.243848 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1569a80-47e9-45a8-8ba9-2d399719f292 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:44.244125 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1569a80-47e9-45a8-8ba9-2d399719f292 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:44.244399 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1569a80-47e9-45a8-8ba9-2d399719f292 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:44.245106 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1569a80-47e9-45a8-8ba9-2d399719f292 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:16:44.246530 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1569a80-47e9-45a8-8ba9-2d399719f292 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:16:44.271589 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 599/1205] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 5 10:16:44 2022] POST /identity/v3/users => generated 251 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:16:46.584058 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0761150-2c56-4cbf-a3db-d54001aaf92f None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:46.584663 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0761150-2c56-4cbf-a3db-d54001aaf92f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:46.584663 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0761150-2c56-4cbf-a3db-d54001aaf92f None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:46.585979 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 607/1206] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:16:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:16:46.591651 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b2849-3185-4e1e-8efd-6aa338f5697d None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:46.592086 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b2849-3185-4e1e-8efd-6aa338f5697d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:46.592188 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b2849-3185-4e1e-8efd-6aa338f5697d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:46.612906 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b42b2849-3185-4e1e-8efd-6aa338f5697d None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:16:46.614353 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b42b2849-3185-4e1e-8efd-6aa338f5697d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:16:46.707341 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 600/1207] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:16:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:16:46.731245 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0761150-2c56-4cbf-a3db-d54001aaf92f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:16:46.850582 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0761150-2c56-4cbf-a3db-d54001aaf92f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:16:46.893184 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80491987-2ad1-4195-a3e8-4079999d10de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:16:46.893962 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80491987-2ad1-4195-a3e8-4079999d10de None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:46.894709 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80491987-2ad1-4195-a3e8-4079999d10de None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:46.895021 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80491987-2ad1-4195-a3e8-4079999d10de None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:46.909173 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80491987-2ad1-4195-a3e8-4079999d10de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:16:46.915271 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-80491987-2ad1-4195-a3e8-4079999d10de None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 10:16:46.915271 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:16:46.915271 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:16:46.915271 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:16:46.915271 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:16:46.915271 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:16:46.915271 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:16:46.915271 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:16:46.915271 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:16:46.915271 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:16:46.915271 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:16:46.915271 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:16:46.915271 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:16:46.915271 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:16:46.915271 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:16:46.915271 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:16:46.916318 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:16:46.916318 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:16:46.916318 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:16:46.916318 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:16:46.916318 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:16:46.916318 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:16:46.916318 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:16:46.916318 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:16:46.916318 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:16:46.916318 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:16:46.916318 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:16:46.916318 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:16:46.916318 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:16:46.916318 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:16:46.916318 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:16:46.916318 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:16:46.917220 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:16:46.917220 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:16:46.917220 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:16:46.917220 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:16:46.917220 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:16:46.917220 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:16:46.917220 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:16:46.917220 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:16:46.917220 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:16:46.917220 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:16:46.917220 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 10:16:46.917220 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:16:46.921844 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 608/1208] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 10:16:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:16:46.922960 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b2849-3185-4e1e-8efd-6aa338f5697d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:16:46.970106 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:16:47.039671 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b2849-3185-4e1e-8efd-6aa338f5697d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:16:47.078942 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ee85b-51d4-42fb-8074-254965a78338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:16:47.079431 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ee85b-51d4-42fb-8074-254965a78338 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:47.079714 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ee85b-51d4-42fb-8074-254965a78338 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:47.080001 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ee85b-51d4-42fb-8074-254965a78338 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:47.080795 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08ee85b-51d4-42fb-8074-254965a78338 None admin] RBAC: Adding query filter params (name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:16:47.081100 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08ee85b-51d4-42fb-8074-254965a78338 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:16:47.082832 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08ee85b-51d4-42fb-8074-254965a78338 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:16:47.092268 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 601/1209] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 5 10:16:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:16:47.098213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80491987-2ad1-4195-a3e8-4079999d10de None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:16:47.164095 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:16:47.227948 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80491987-2ad1-4195-a3e8-4079999d10de None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:16:47.269253 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ba42c-a6dc-4ff7-973a-95e5f1c2b3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:16:47.269961 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ba42c-a6dc-4ff7-973a-95e5f1c2b3b3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:47.270245 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ba42c-a6dc-4ff7-973a-95e5f1c2b3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:47.270522 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ba42c-a6dc-4ff7-973a-95e5f1c2b3b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:47.283312 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ba42c-a6dc-4ff7-973a-95e5f1c2b3b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:16:47.285216 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ba42c-a6dc-4ff7-973a-95e5f1c2b3b3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:16:47.307764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-1a0ba42c-a6dc-4ff7-973a-95e5f1c2b3b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:16:47.307764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:16:47.307764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:16:47.307764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:16:47.307764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:16:47.307764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:16:47.307764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:16:47.307764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:16:47.307764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:16:47.307764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:16:47.307764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:16:47.307764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:16:47.307764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:16:47.307764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:16:47.307764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:16:47.308852 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:16:47.308852 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:16:47.308852 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:16:47.308852 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:16:47.308852 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:16:47.308852 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:16:47.308852 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:16:47.308852 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:16:47.308852 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:16:47.308852 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:16:47.308852 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:16:47.308852 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:16:47.308852 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:16:47.308852 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:16:47.308852 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:16:47.308852 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:16:47.308852 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:16:47.309927 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:16:47.309927 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:16:47.309927 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:16:47.309927 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:16:47.309927 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 609/1210] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:16:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:16:47.318929 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ee85b-51d4-42fb-8074-254965a78338 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:16:47.434977 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:16:47.438930 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ee85b-51d4-42fb-8074-254965a78338 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:16:47.477080 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d335976-f285-45cd-a89b-7424c8a45437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:16:47.477785 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d335976-f285-45cd-a89b-7424c8a45437 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:47.478139 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d335976-f285-45cd-a89b-7424c8a45437 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:47.478430 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d335976-f285-45cd-a89b-7424c8a45437 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:47.479222 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d335976-f285-45cd-a89b-7424c8a45437 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:16:47.479547 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d335976-f285-45cd-a89b-7424c8a45437 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:16:47.481130 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d335976-f285-45cd-a89b-7424c8a45437 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:16:47.494637 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 602/1211] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:16:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:16:47.500372 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ba42c-a6dc-4ff7-973a-95e5f1c2b3b3 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:16:47.623653 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ba42c-a6dc-4ff7-973a-95e5f1c2b3b3 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:16:47.625436 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:16:47.664644 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9861227-1aea-4de4-9b7f-128bffee55f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:16:47.665300 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9861227-1aea-4de4-9b7f-128bffee55f4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:47.665584 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9861227-1aea-4de4-9b7f-128bffee55f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:47.665866 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9861227-1aea-4de4-9b7f-128bffee55f4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:47.675497 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9861227-1aea-4de4-9b7f-128bffee55f4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:16:47.677198 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9861227-1aea-4de4-9b7f-128bffee55f4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:16:47.690404 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-d9861227-1aea-4de4-9b7f-128bffee55f4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 05 10:16:47.690404 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:16:47.690404 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:16:47.690404 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:16:47.690404 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:16:47.690404 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:16:47.690404 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:16:47.690404 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:16:47.690404 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:16:47.690404 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:16:47.690404 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:16:47.690404 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:16:47.690404 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 10:16:47.690404 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 10:16:47.690404 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 10:16:47.690404 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 10:16:47.697312 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:16:47.697312 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:16:47.697312 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 10:16:47.697312 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:16:47.697312 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 10:16:47.697312 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:16:47.697312 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:16:47.697312 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:16:47.697312 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:16:47.697312 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:16:47.697312 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:16:47.697312 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:16:47.697312 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:16:47.697312 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:16:47.697312 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:16:47.697312 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:16:47.697312 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:16:47.697312 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:16:47.698550 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:16:47.698550 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:16:47.698550 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 10:16:47.698550 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 10:16:47.698550 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 10:16:47.698550 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 10:16:47.698550 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 10:16:47.698550 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 10:16:47.698550 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 05 10:16:47.698550 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:16:47.698550 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 610/1212] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:16:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:16:47.699291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d335976-f285-45cd-a89b-7424c8a45437 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:16:47.811358 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d335976-f285-45cd-a89b-7424c8a45437 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:16:47.851272 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb9d04e-f03e-4ac1-a02d-3da118029741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:16:47.852069 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9d04e-f03e-4ac1-a02d-3da118029741 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:47.852330 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9d04e-f03e-4ac1-a02d-3da118029741 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:47.852715 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9d04e-f03e-4ac1-a02d-3da118029741 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:47.853516 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb9d04e-f03e-4ac1-a02d-3da118029741 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:16:47.853940 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb9d04e-f03e-4ac1-a02d-3da118029741 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:16:47.855691 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb9d04e-f03e-4ac1-a02d-3da118029741 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:16:47.876945 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 603/1213] 10.222.0.40 () {56 vars in 1143 bytes} [Fri Aug 5 10:16:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:16:47.883822 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9861227-1aea-4de4-9b7f-128bffee55f4 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:16:48.006288 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9861227-1aea-4de4-9b7f-128bffee55f4 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:16:48.046504 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b591e9-22a5-474d-9291-a2eb66835f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:16:48.047408 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b591e9-22a5-474d-9291-a2eb66835f04 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:48.047786 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b591e9-22a5-474d-9291-a2eb66835f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:48.048209 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b591e9-22a5-474d-9291-a2eb66835f04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:48.079433 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:16:48.128092 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b591e9-22a5-474d-9291-a2eb66835f04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:16:48.130175 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b591e9-22a5-474d-9291-a2eb66835f04 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:16:48.267278 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 611/1214] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:16:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 05 10:16:48.273238 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb9d04e-f03e-4ac1-a02d-3da118029741 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:16:48.299639 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:16:48.406048 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb9d04e-f03e-4ac1-a02d-3da118029741 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:16:48.450687 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd01ce-7da0-4676-91e9-2163acec922d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:16:48.451371 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd01ce-7da0-4676-91e9-2163acec922d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:48.451688 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd01ce-7da0-4676-91e9-2163acec922d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:48.451919 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd01ce-7da0-4676-91e9-2163acec922d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:48.463444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dd01ce-7da0-4676-91e9-2163acec922d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:16:48.465043 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dd01ce-7da0-4676-91e9-2163acec922d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:16:48.481576 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-38dd01ce-7da0-4676-91e9-2163acec922d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:16:48.481576 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:16:48.481576 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:16:48.481576 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:16:48.481576 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:16:48.481576 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:16:48.481576 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:16:48.481576 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:16:48.481576 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:16:48.481576 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:16:48.481576 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:16:48.481576 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:16:48.481576 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:16:48.481576 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:16:48.481576 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:16:48.482760 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:16:48.482760 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:16:48.482760 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:16:48.482760 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:16:48.482760 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:16:48.482760 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:16:48.482760 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:16:48.482760 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:16:48.482760 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:16:48.482760 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:16:48.482760 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:16:48.482760 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:16:48.482760 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:16:48.482760 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:16:48.482760 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:16:48.482760 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:16:48.482760 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:16:48.483905 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:16:48.483905 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:16:48.483905 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:16:48.483905 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:16:48.484572 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 604/1215] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:16:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:16:48.493591 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b591e9-22a5-474d-9291-a2eb66835f04 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:16:48.567678 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:16:48.612270 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b591e9-22a5-474d-9291-a2eb66835f04 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:16:48.651031 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbd20f4-1c9f-4363-8e9c-ecd2627090c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:16:48.652504 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd20f4-1c9f-4363-8e9c-ecd2627090c4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:48.652748 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd20f4-1c9f-4363-8e9c-ecd2627090c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:48.653071 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd20f4-1c9f-4363-8e9c-ecd2627090c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:48.653729 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbd20f4-1c9f-4363-8e9c-ecd2627090c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:16:48.654033 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbd20f4-1c9f-4363-8e9c-ecd2627090c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:16:48.655475 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbd20f4-1c9f-4363-8e9c-ecd2627090c4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:16:48.676250 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 612/1216] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:16:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:16:48.682592 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd01ce-7da0-4676-91e9-2163acec922d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:16:48.756771 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:16:48.791411 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd01ce-7da0-4676-91e9-2163acec922d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:16:48.828521 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0e34a4-5c65-4efc-b139-9d57e07a832f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:16:48.829269 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e34a4-5c65-4efc-b139-9d57e07a832f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:48.829707 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e34a4-5c65-4efc-b139-9d57e07a832f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:48.829992 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e34a4-5c65-4efc-b139-9d57e07a832f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:48.839817 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0e34a4-5c65-4efc-b139-9d57e07a832f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:16:48.841409 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0e34a4-5c65-4efc-b139-9d57e07a832f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:16:48.853618 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-cd0e34a4-5c65-4efc-b139-9d57e07a832f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:16:48.853618 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:16:48.853618 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:16:48.853618 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:16:48.853618 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:16:48.853618 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:16:48.853618 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:16:48.853618 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:16:48.853618 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:16:48.853618 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:16:48.853618 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:16:48.853618 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:16:48.853618 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 10:16:48.853618 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 10:16:48.853618 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 10:16:48.854781 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 10:16:48.854781 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:16:48.854781 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:16:48.854781 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:16:48.854781 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:16:48.854781 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:16:48.854781 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:16:48.854781 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:16:48.854781 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:16:48.854781 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:16:48.854781 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:16:48.854781 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:16:48.854781 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:16:48.854781 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:16:48.854781 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:16:48.854781 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:16:48.854781 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:16:48.860464 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 10:16:48.860464 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 10:16:48.860464 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 10:16:48.860464 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 10:16:48.860464 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 10:16:48.860464 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 10:16:48.860464 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:16:48.860464 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:16:48.860464 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 605/1217] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:16:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:16:48.860969 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbd20f4-1c9f-4363-8e9c-ecd2627090c4 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:16:48.980100 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbd20f4-1c9f-4363-8e9c-ecd2627090c4 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:16:49.006361 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:16:49.019536 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa999a-789a-403a-bc91-77b58542a965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:16:49.020974 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa999a-789a-403a-bc91-77b58542a965 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:49.020974 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa999a-789a-403a-bc91-77b58542a965 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:49.020974 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa999a-789a-403a-bc91-77b58542a965 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:49.021702 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daa999a-789a-403a-bc91-77b58542a965 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:16:49.021906 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daa999a-789a-403a-bc91-77b58542a965 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:16:49.023510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daa999a-789a-403a-bc91-77b58542a965 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:16:49.042562 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 613/1218] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 10:16:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:16:49.043645 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0e34a4-5c65-4efc-b139-9d57e07a832f None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:16:49.156090 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0e34a4-5c65-4efc-b139-9d57e07a832f None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:16:49.188082 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:16:49.195645 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b963a2-1386-41a8-b6dd-3b770e9f8990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:16:49.196269 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b963a2-1386-41a8-b6dd-3b770e9f8990 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:49.196594 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b963a2-1386-41a8-b6dd-3b770e9f8990 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:49.196864 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b963a2-1386-41a8-b6dd-3b770e9f8990 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:49.197696 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b963a2-1386-41a8-b6dd-3b770e9f8990 None admin] RBAC: Adding query filter params (role.id=3a2b4319820a488198dabee34420cdc7, scope.project.id=36b8b505012d4d0bac54845b6799e743, user.id=8743f5bb0be54d57a957617636b0b8fe) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:16:49.198004 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b963a2-1386-41a8-b6dd-3b770e9f8990 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:16:49.199492 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b963a2-1386-41a8-b6dd-3b770e9f8990 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:16:49.214936 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 606/1219] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:16:49 2022] GET /identity/v3/role_assignments?user.id=8743f5bb0be54d57a957617636b0b8fe&scope.project.id=36b8b505012d4d0bac54845b6799e743&role.id=3a2b4319820a488198dabee34420cdc7 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:16:51.265405 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3959940-681c-4a1f-90d6-9b63ed61526f None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:51.266060 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3959940-681c-4a1f-90d6-9b63ed61526f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:51.266060 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3959940-681c-4a1f-90d6-9b63ed61526f None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:51.267717 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 614/1220] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:16:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:16:51.273251 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b135a-b354-44d2-b818-69cee3850f3f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:51.273801 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b135a-b354-44d2-b818-69cee3850f3f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:51.273801 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b135a-b354-44d2-b818-69cee3850f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:51.293137 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-f13b135a-b354-44d2-b818-69cee3850f3f None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:16:51.294418 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f13b135a-b354-44d2-b818-69cee3850f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:16:51.362816 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 607/1221] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:16:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:16:51.385085 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3959940-681c-4a1f-90d6-9b63ed61526f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:16:51.496211 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3959940-681c-4a1f-90d6-9b63ed61526f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:16:51.531098 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ce9f4-8642-457f-b65a-877b9ead35f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:16:51.531741 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ce9f4-8642-457f-b65a-877b9ead35f0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:51.532014 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ce9f4-8642-457f-b65a-877b9ead35f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:51.532280 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ce9f4-8642-457f-b65a-877b9ead35f0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:51.542535 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08ce9f4-8642-457f-b65a-877b9ead35f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:16:51.547445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-e08ce9f4-8642-457f-b65a-877b9ead35f0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 10:16:51.547445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:16:51.547445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:16:51.547445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:16:51.547445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:16:51.547445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:16:51.547445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:16:51.547445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:16:51.547445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:16:51.547445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:16:51.547445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:16:51.547445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:16:51.547445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:16:51.547445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:16:51.547445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:16:51.547445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:16:51.548493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:16:51.548493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:16:51.548493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:16:51.548493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:16:51.548493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:16:51.548493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:16:51.548493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:16:51.548493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:16:51.548493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:16:51.548493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:16:51.548493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:16:51.548493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:16:51.548493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:16:51.548493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:16:51.548493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:16:51.548493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:16:51.549351 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:16:51.549351 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:16:51.549351 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:16:51.549351 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:16:51.549351 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:16:51.549351 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:16:51.549351 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:16:51.549351 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:16:51.549351 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:16:51.549351 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:16:51.549351 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 10:16:51.549351 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:16:51.553746 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 615/1222] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 10:16:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:16:51.554733 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13b135a-b354-44d2-b818-69cee3850f3f None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:16:51.656710 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13b135a-b354-44d2-b818-69cee3850f3f None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:16:51.684901 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:16:51.696684 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de54ce-1de9-4dd7-9b3a-161464f27f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:16:51.697310 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de54ce-1de9-4dd7-9b3a-161464f27f36 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:51.697590 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de54ce-1de9-4dd7-9b3a-161464f27f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:51.697865 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de54ce-1de9-4dd7-9b3a-161464f27f36 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:51.698589 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0de54ce-1de9-4dd7-9b3a-161464f27f36 None admin] RBAC: Adding query filter params (name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:16:51.698894 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0de54ce-1de9-4dd7-9b3a-161464f27f36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:16:51.700323 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0de54ce-1de9-4dd7-9b3a-161464f27f36 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:16:51.709734 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 608/1223] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 5 10:16:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:16:51.716140 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ce9f4-8642-457f-b65a-877b9ead35f0 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:16:51.766760 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:16:51.827992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ce9f4-8642-457f-b65a-877b9ead35f0 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:16:51.863245 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0f8aae-00ad-43a1-ae5b-c46c16582bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:16:51.863884 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f8aae-00ad-43a1-ae5b-c46c16582bc9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:51.864165 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f8aae-00ad-43a1-ae5b-c46c16582bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:51.864423 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f8aae-00ad-43a1-ae5b-c46c16582bc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:51.875297 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0f8aae-00ad-43a1-ae5b-c46c16582bc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:16:51.876760 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0f8aae-00ad-43a1-ae5b-c46c16582bc9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:16:51.892602 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-9f0f8aae-00ad-43a1-ae5b-c46c16582bc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:16:51.892602 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:16:51.892602 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:16:51.892602 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:16:51.892602 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:16:51.892602 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:16:51.892602 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:16:51.892602 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:16:51.892602 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:16:51.892602 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:16:51.892602 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:16:51.892602 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:16:51.892602 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:16:51.892602 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:16:51.892602 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:16:51.893759 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:16:51.893759 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:16:51.893759 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:16:51.893759 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:16:51.893759 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:16:51.893759 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:16:51.893759 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:16:51.893759 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:16:51.893759 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:16:51.893759 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:16:51.893759 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:16:51.893759 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:16:51.893759 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:16:51.893759 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:16:51.893759 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:16:51.893759 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:16:51.893759 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:16:51.894764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:16:51.894764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:16:51.894764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:16:51.894764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:16:51.894764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 616/1224] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:16:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:16:51.900907 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de54ce-1de9-4dd7-9b3a-161464f27f36 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:16:52.008930 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de54ce-1de9-4dd7-9b3a-161464f27f36 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:16:52.047703 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7ca302-120c-4837-9084-2fcdf2643591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:16:52.048228 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ca302-120c-4837-9084-2fcdf2643591 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:52.048501 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ca302-120c-4837-9084-2fcdf2643591 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:52.048779 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ca302-120c-4837-9084-2fcdf2643591 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:52.049460 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7ca302-120c-4837-9084-2fcdf2643591 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:16:52.049761 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7ca302-120c-4837-9084-2fcdf2643591 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:16:52.051352 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7ca302-120c-4837-9084-2fcdf2643591 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:16:52.063374 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 609/1225] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:16:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:16:52.069371 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0f8aae-00ad-43a1-ae5b-c46c16582bc9 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:16:52.169831 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:16:52.177417 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0f8aae-00ad-43a1-ae5b-c46c16582bc9 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:16:52.211506 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db16826-52a0-4516-8052-d7e31e270b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:16:52.212145 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db16826-52a0-4516-8052-d7e31e270b7e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:52.212419 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db16826-52a0-4516-8052-d7e31e270b7e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:52.212688 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db16826-52a0-4516-8052-d7e31e270b7e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:52.220116 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db16826-52a0-4516-8052-d7e31e270b7e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:16:52.221568 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db16826-52a0-4516-8052-d7e31e270b7e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:16:52.233082 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-6db16826-52a0-4516-8052-d7e31e270b7e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 05 10:16:52.233082 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:16:52.233082 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:16:52.233082 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:16:52.233082 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:16:52.233082 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:16:52.233082 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:16:52.233082 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:16:52.233082 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:16:52.233082 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:16:52.233082 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:16:52.233082 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:16:52.233082 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 10:16:52.233082 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 10:16:52.233082 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 10:16:52.233082 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 10:16:52.234246 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:16:52.234246 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:16:52.234246 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 10:16:52.234246 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:16:52.234246 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 10:16:52.234246 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:16:52.234246 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:16:52.234246 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:16:52.234246 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:16:52.234246 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:16:52.234246 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:16:52.234246 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:16:52.234246 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:16:52.234246 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:16:52.234246 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:16:52.234246 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:16:52.234246 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:16:52.234246 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:16:52.235245 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:16:52.235245 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:16:52.235245 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 10:16:52.235245 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 10:16:52.235245 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 10:16:52.235245 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 10:16:52.235245 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 10:16:52.235245 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 10:16:52.235245 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 05 10:16:52.235245 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:16:52.235245 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 617/1226] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:16:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:16:52.240218 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7ca302-120c-4837-9084-2fcdf2643591 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:16:52.290236 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:16:52.353424 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7ca302-120c-4837-9084-2fcdf2643591 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:16:52.390095 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f54c237-0408-43a3-bce1-5402fa25aa92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:16:52.390871 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f54c237-0408-43a3-bce1-5402fa25aa92 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:52.391153 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f54c237-0408-43a3-bce1-5402fa25aa92 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:52.391420 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f54c237-0408-43a3-bce1-5402fa25aa92 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:52.392222 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f54c237-0408-43a3-bce1-5402fa25aa92 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:16:52.392526 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f54c237-0408-43a3-bce1-5402fa25aa92 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:16:52.394628 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f54c237-0408-43a3-bce1-5402fa25aa92 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:16:52.412899 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 610/1227] 10.222.0.40 () {56 vars in 1143 bytes} [Fri Aug 5 10:16:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:16:52.419809 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db16826-52a0-4516-8052-d7e31e270b7e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:16:52.529564 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db16826-52a0-4516-8052-d7e31e270b7e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:16:52.565060 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b3826-2a3e-4d60-9e49-e051424e18c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:16:52.565764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b3826-2a3e-4d60-9e49-e051424e18c1 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:52.566044 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b3826-2a3e-4d60-9e49-e051424e18c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:52.566313 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b3826-2a3e-4d60-9e49-e051424e18c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:52.577145 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7b3826-2a3e-4d60-9e49-e051424e18c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:16:52.578606 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7b3826-2a3e-4d60-9e49-e051424e18c1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:16:52.593930 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-9b7b3826-2a3e-4d60-9e49-e051424e18c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:16:52.593930 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:16:52.593930 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:16:52.593930 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:16:52.593930 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:16:52.593930 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:16:52.593930 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:16:52.593930 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:16:52.593930 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:16:52.593930 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:16:52.593930 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:16:52.593930 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:16:52.593930 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:16:52.593930 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:16:52.593930 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:16:52.595078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:16:52.595078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:16:52.595078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:16:52.595078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:16:52.595078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:16:52.595078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:16:52.595078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:16:52.595078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:16:52.595078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:16:52.595078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:16:52.595078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:16:52.595078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:16:52.595078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:16:52.595078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:16:52.595078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:16:52.595078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:16:52.595078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:16:52.596160 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:16:52.596160 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:16:52.596160 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:16:52.596160 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:16:52.596160 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 618/1228] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:16:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:16:52.601487 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f54c237-0408-43a3-bce1-5402fa25aa92 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:16:52.709940 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f54c237-0408-43a3-bce1-5402fa25aa92 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:16:52.712304 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:16:52.748253 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3846934-06b7-40b1-8244-8aa617b59945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:16:52.748946 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3846934-06b7-40b1-8244-8aa617b59945 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:52.749261 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3846934-06b7-40b1-8244-8aa617b59945 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:52.749543 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3846934-06b7-40b1-8244-8aa617b59945 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:52.750303 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3846934-06b7-40b1-8244-8aa617b59945 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:16:52.750695 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3846934-06b7-40b1-8244-8aa617b59945 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:16:52.752356 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3846934-06b7-40b1-8244-8aa617b59945 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:16:52.766001 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 611/1229] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:16:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:16:52.772686 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b3826-2a3e-4d60-9e49-e051424e18c1 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:16:52.782392 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:16:52.887594 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b3826-2a3e-4d60-9e49-e051424e18c1 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:16:52.926771 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b7c780-a760-4d26-a4ef-d12c2069eef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:16:52.927448 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b7c780-a760-4d26-a4ef-d12c2069eef1 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:52.927729 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b7c780-a760-4d26-a4ef-d12c2069eef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:52.928000 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b7c780-a760-4d26-a4ef-d12c2069eef1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:52.934961 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b7c780-a760-4d26-a4ef-d12c2069eef1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:16:52.936473 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b7c780-a760-4d26-a4ef-d12c2069eef1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:16:52.949252 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-c0b7c780-a760-4d26-a4ef-d12c2069eef1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:16:52.949252 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:16:52.949252 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:16:52.949252 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:16:52.949252 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:16:52.949252 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:16:52.949252 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:16:52.949252 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:16:52.949252 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:16:52.949252 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:16:52.949252 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:16:52.949252 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:16:52.949252 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 10:16:52.949252 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 10:16:52.949252 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 10:16:52.950389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 10:16:52.950389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:16:52.950389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:16:52.950389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:16:52.950389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:16:52.950389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:16:52.950389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:16:52.950389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:16:52.950389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:16:52.950389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:16:52.950389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:16:52.950389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:16:52.950389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:16:52.950389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:16:52.950389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:16:52.950389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:16:52.950389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:16:52.951337 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 10:16:52.951337 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 10:16:52.951337 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 10:16:52.951337 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 10:16:52.951337 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 10:16:52.951337 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 10:16:52.951337 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:16:52.951337 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:16:52.951337 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 619/1230] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:16:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:16:52.956689 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3846934-06b7-40b1-8244-8aa617b59945 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:16:53.073147 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3846934-06b7-40b1-8244-8aa617b59945 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:16:53.107147 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e03231e-b2da-43fe-b388-836b3df7f7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:16:53.107773 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03231e-b2da-43fe-b388-836b3df7f7ab None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:53.108096 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03231e-b2da-43fe-b388-836b3df7f7ab None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:53.108327 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03231e-b2da-43fe-b388-836b3df7f7ab None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:53.109068 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e03231e-b2da-43fe-b388-836b3df7f7ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:16:53.109382 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e03231e-b2da-43fe-b388-836b3df7f7ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:16:53.110869 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e03231e-b2da-43fe-b388-836b3df7f7ab None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:16:53.124853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 612/1231] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 10:16:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:16:53.131324 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b7c780-a760-4d26-a4ef-d12c2069eef1 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:16:53.206425 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:16:53.242673 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b7c780-a760-4d26-a4ef-d12c2069eef1 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:16:53.281511 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bcdb1a-b3ae-41e0-8ac2-bfbefef20c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:16:53.282276 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bcdb1a-b3ae-41e0-8ac2-bfbefef20c95 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:53.282538 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bcdb1a-b3ae-41e0-8ac2-bfbefef20c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:53.282830 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bcdb1a-b3ae-41e0-8ac2-bfbefef20c95 None admin] PATH_INFO: `/v3/projects/36b8b505012d4d0bac54845b6799e743/users/8743f5bb0be54d57a957617636b0b8fe/roles/3a2b4319820a488198dabee34420cdc7` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:53.299577 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bcdb1a-b3ae-41e0-8ac2-bfbefef20c95 None admin] RBAC: Authorizing `identity:create_grant(project_id=36b8b505012d4d0bac54845b6799e743, user_id=8743f5bb0be54d57a957617636b0b8fe, role_id=3a2b4319820a488198dabee34420cdc7)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:16:53.301583 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bcdb1a-b3ae-41e0-8ac2-bfbefef20c95 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:16:53.315516 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 620/1232] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 10:16:53 2022] PUT /identity/v3/projects/36b8b505012d4d0bac54845b6799e743/users/8743f5bb0be54d57a957617636b0b8fe/roles/3a2b4319820a488198dabee34420cdc7 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:16:55.368597 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48ad2d-feae-424b-93c3-8062dc052ac2 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:55.369203 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48ad2d-feae-424b-93c3-8062dc052ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:55.369322 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48ad2d-feae-424b-93c3-8062dc052ac2 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:55.370936 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 613/1233] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:16:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:16:55.377286 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e4047f-85ec-417f-8354-5b6f21b61378 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:55.377802 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e4047f-85ec-417f-8354-5b6f21b61378 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:55.377999 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e4047f-85ec-417f-8354-5b6f21b61378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:55.398624 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-62e4047f-85ec-417f-8354-5b6f21b61378 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:16:55.399998 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-62e4047f-85ec-417f-8354-5b6f21b61378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:16:55.473450 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 621/1234] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:16:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:16:55.496101 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af48ad2d-feae-424b-93c3-8062dc052ac2 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:16:55.522017 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:16:55.606694 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af48ad2d-feae-424b-93c3-8062dc052ac2 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:16:55.646306 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd5ed9-5a67-4323-9b19-9abcd4e7c40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:16:55.646976 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd5ed9-5a67-4323-9b19-9abcd4e7c40b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:55.647255 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd5ed9-5a67-4323-9b19-9abcd4e7c40b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:55.647542 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd5ed9-5a67-4323-9b19-9abcd4e7c40b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:55.661836 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdd5ed9-5a67-4323-9b19-9abcd4e7c40b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:16:55.666774 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-1fdd5ed9-5a67-4323-9b19-9abcd4e7c40b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 10:16:55.666774 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:16:55.666774 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:16:55.666774 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:16:55.666774 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:16:55.666774 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:16:55.666774 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:16:55.666774 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:16:55.666774 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:16:55.666774 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:16:55.666774 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:16:55.666774 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:16:55.666774 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:16:55.666774 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:16:55.666774 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:16:55.666774 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:16:55.668027 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:16:55.668027 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:16:55.668027 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:16:55.668027 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:16:55.668027 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:16:55.668027 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:16:55.668027 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:16:55.668027 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:16:55.668027 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:16:55.668027 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:16:55.668027 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:16:55.668027 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:16:55.668027 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:16:55.668027 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:16:55.668027 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:16:55.668027 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:16:55.669331 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:16:55.669331 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:16:55.669331 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:16:55.669331 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:16:55.669331 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:16:55.669331 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:16:55.669331 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:16:55.669331 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:16:55.669331 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:16:55.669331 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:16:55.669331 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 10:16:55.669331 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:16:55.669331 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 614/1235] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 10:16:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:16:55.674474 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e4047f-85ec-417f-8354-5b6f21b61378 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:16:55.720049 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:16:55.781049 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e4047f-85ec-417f-8354-5b6f21b61378 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:16:55.820946 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29399d59-093d-495d-8312-653342632915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:16:55.821650 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29399d59-093d-495d-8312-653342632915 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:55.821903 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29399d59-093d-495d-8312-653342632915 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:55.822213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29399d59-093d-495d-8312-653342632915 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:55.823010 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29399d59-093d-495d-8312-653342632915 None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:16:55.823318 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29399d59-093d-495d-8312-653342632915 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:16:55.824798 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29399d59-093d-495d-8312-653342632915 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:16:55.833744 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 622/1236] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 5 10:16:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:16:55.841303 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd5ed9-5a67-4323-9b19-9abcd4e7c40b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:16:55.944643 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:16:55.964080 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd5ed9-5a67-4323-9b19-9abcd4e7c40b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:16:55.999560 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17518bc0-ce26-42aa-863a-d130971d24e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:16:56.000220 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17518bc0-ce26-42aa-863a-d130971d24e4 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:56.000529 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17518bc0-ce26-42aa-863a-d130971d24e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:56.000804 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17518bc0-ce26-42aa-863a-d130971d24e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:56.011818 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17518bc0-ce26-42aa-863a-d130971d24e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:16:56.013426 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17518bc0-ce26-42aa-863a-d130971d24e4 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:16:56.028535 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-17518bc0-ce26-42aa-863a-d130971d24e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:16:56.028535 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:16:56.028535 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:16:56.028535 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:16:56.028535 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:16:56.028535 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:16:56.028535 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:16:56.028535 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:16:56.028535 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:16:56.028535 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:16:56.028535 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:16:56.028535 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:16:56.028535 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:16:56.028535 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:16:56.028535 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:16:56.029615 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:16:56.029615 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:16:56.029615 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:16:56.029615 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:16:56.029615 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:16:56.029615 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:16:56.029615 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:16:56.029615 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:16:56.029615 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:16:56.029615 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:16:56.029615 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:16:56.029615 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:16:56.029615 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:16:56.029615 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:16:56.029615 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:16:56.029615 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:16:56.029615 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:16:56.030733 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:16:56.030733 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:16:56.030733 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:16:56.030733 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:16:56.030733 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 615/1237] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:16:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:16:56.037924 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29399d59-093d-495d-8312-653342632915 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:16:56.140167 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29399d59-093d-495d-8312-653342632915 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:16:56.149488 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:16:56.180947 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77943523-703e-4d56-b2b9-0263872d4514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:16:56.181620 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77943523-703e-4d56-b2b9-0263872d4514 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:56.181939 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77943523-703e-4d56-b2b9-0263872d4514 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:56.182215 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77943523-703e-4d56-b2b9-0263872d4514 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:56.183112 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77943523-703e-4d56-b2b9-0263872d4514 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:16:56.183477 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77943523-703e-4d56-b2b9-0263872d4514 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:16:56.184907 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77943523-703e-4d56-b2b9-0263872d4514 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:16:56.197620 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 623/1238] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:16:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:16:56.203593 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17518bc0-ce26-42aa-863a-d130971d24e4 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:16:56.308874 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17518bc0-ce26-42aa-863a-d130971d24e4 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:16:56.343100 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214d942c-0015-443f-bbdb-2314b878ab50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:16:56.343751 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d942c-0015-443f-bbdb-2314b878ab50 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:56.344032 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d942c-0015-443f-bbdb-2314b878ab50 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:56.344306 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d942c-0015-443f-bbdb-2314b878ab50 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:56.347254 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:16:56.356538 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214d942c-0015-443f-bbdb-2314b878ab50 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:16:56.356538 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214d942c-0015-443f-bbdb-2314b878ab50 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:16:56.368658 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-214d942c-0015-443f-bbdb-2314b878ab50 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 05 10:16:56.368658 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:16:56.368658 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:16:56.368658 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:16:56.368658 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:16:56.368658 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:16:56.368658 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:16:56.368658 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:16:56.368658 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:16:56.368658 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:16:56.368658 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:16:56.368658 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:16:56.368658 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 10:16:56.368658 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 10:16:56.368658 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 10:16:56.368658 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 10:16:56.370283 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:16:56.370283 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:16:56.370283 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 10:16:56.370283 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:16:56.370283 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 10:16:56.370283 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:16:56.370283 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:16:56.370283 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:16:56.370283 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:16:56.370283 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:16:56.370283 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:16:56.370283 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:16:56.370283 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:16:56.370283 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:16:56.370283 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:16:56.370283 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:16:56.370283 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:16:56.370283 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:16:56.371281 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:16:56.371281 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:16:56.371281 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 10:16:56.371281 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 10:16:56.371281 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 10:16:56.371281 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 10:16:56.371281 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 10:16:56.371281 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 10:16:56.371281 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 05 10:16:56.371281 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:16:56.371281 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 616/1239] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:16:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:16:56.375860 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77943523-703e-4d56-b2b9-0263872d4514 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:16:56.485246 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77943523-703e-4d56-b2b9-0263872d4514 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:16:56.527747 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d004db20-f940-4b3a-ac7b-ed6e2fe443f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:16:56.528381 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004db20-f940-4b3a-ac7b-ed6e2fe443f8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:56.528681 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004db20-f940-4b3a-ac7b-ed6e2fe443f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:56.528917 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004db20-f940-4b3a-ac7b-ed6e2fe443f8 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:56.529801 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d004db20-f940-4b3a-ac7b-ed6e2fe443f8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:16:56.530047 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d004db20-f940-4b3a-ac7b-ed6e2fe443f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:16:56.531558 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d004db20-f940-4b3a-ac7b-ed6e2fe443f8 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:16:56.549646 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 624/1240] 10.222.0.40 () {56 vars in 1143 bytes} [Fri Aug 5 10:16:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:16:56.555854 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214d942c-0015-443f-bbdb-2314b878ab50 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:16:56.663832 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214d942c-0015-443f-bbdb-2314b878ab50 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:16:56.699492 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04df05d-1aa3-409e-8fe8-78d9f3131b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:16:56.700123 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04df05d-1aa3-409e-8fe8-78d9f3131b73 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:56.700400 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04df05d-1aa3-409e-8fe8-78d9f3131b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:56.700670 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04df05d-1aa3-409e-8fe8-78d9f3131b73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:56.769196 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:16:56.776470 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04df05d-1aa3-409e-8fe8-78d9f3131b73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:16:56.778034 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04df05d-1aa3-409e-8fe8-78d9f3131b73 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:16:56.887797 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 617/1241] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:16:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 10:16:56.894397 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d004db20-f940-4b3a-ac7b-ed6e2fe443f8 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:16:56.928680 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:16:57.011427 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d004db20-f940-4b3a-ac7b-ed6e2fe443f8 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:16:57.045287 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4b7ece-d460-4d83-9574-4c5f0621caaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:16:57.045945 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b7ece-d460-4d83-9574-4c5f0621caaf None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:57.046201 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b7ece-d460-4d83-9574-4c5f0621caaf None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:57.046466 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b7ece-d460-4d83-9574-4c5f0621caaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:57.056737 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4b7ece-d460-4d83-9574-4c5f0621caaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:16:57.058184 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4b7ece-d460-4d83-9574-4c5f0621caaf None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:16:57.073054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-ca4b7ece-d460-4d83-9574-4c5f0621caaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:16:57.073054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:16:57.073054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:16:57.073054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:16:57.073054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:16:57.073054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:16:57.073054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:16:57.073054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:16:57.073054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:16:57.073054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:16:57.073054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:16:57.073054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:16:57.073054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:16:57.073054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:16:57.073054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:16:57.074120 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:16:57.074120 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:16:57.074120 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:16:57.074120 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:16:57.074120 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:16:57.074120 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:16:57.074120 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:16:57.074120 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:16:57.074120 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:16:57.074120 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:16:57.074120 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:16:57.074120 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:16:57.074120 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:16:57.074120 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:16:57.074120 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:16:57.074120 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:16:57.074120 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:16:57.075107 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:16:57.075107 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:16:57.075107 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:16:57.075107 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:16:57.075107 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 625/1242] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:16:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:16:57.080413 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04df05d-1aa3-409e-8fe8-78d9f3131b73 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:16:57.183170 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:16:57.189825 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04df05d-1aa3-409e-8fe8-78d9f3131b73 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:16:57.225852 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c176f-0d81-4904-8b3b-b9e1bbdfbaea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:16:57.226518 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c176f-0d81-4904-8b3b-b9e1bbdfbaea None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:57.226824 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c176f-0d81-4904-8b3b-b9e1bbdfbaea None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:57.227127 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c176f-0d81-4904-8b3b-b9e1bbdfbaea None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:57.227835 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c176f-0d81-4904-8b3b-b9e1bbdfbaea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:16:57.228126 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c176f-0d81-4904-8b3b-b9e1bbdfbaea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:16:57.229527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c176f-0d81-4904-8b3b-b9e1bbdfbaea None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:16:57.241659 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 618/1243] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:16:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:16:57.247657 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4b7ece-d460-4d83-9574-4c5f0621caaf None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:16:57.336446 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:16:57.364982 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4b7ece-d460-4d83-9574-4c5f0621caaf None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:16:57.401370 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8559793-792c-4f11-abb6-56b52aa83639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:16:57.402067 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8559793-792c-4f11-abb6-56b52aa83639 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:57.402368 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8559793-792c-4f11-abb6-56b52aa83639 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:57.402693 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8559793-792c-4f11-abb6-56b52aa83639 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:57.410744 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8559793-792c-4f11-abb6-56b52aa83639 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:16:57.412332 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8559793-792c-4f11-abb6-56b52aa83639 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:16:57.429136 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-a8559793-792c-4f11-abb6-56b52aa83639 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:16:57.429136 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:16:57.429136 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:16:57.429136 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:16:57.429136 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:16:57.429136 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:16:57.429136 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:16:57.429136 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:16:57.429136 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:16:57.429136 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:16:57.429136 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:16:57.429136 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:16:57.429136 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 10:16:57.429136 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 10:16:57.429136 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 10:16:57.430229 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 10:16:57.430229 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:16:57.430229 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:16:57.430229 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:16:57.430229 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:16:57.430229 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:16:57.430229 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:16:57.430229 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:16:57.430229 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:16:57.430229 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:16:57.430229 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:16:57.430229 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:16:57.430229 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:16:57.430229 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:16:57.430229 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:16:57.430229 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:16:57.430229 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:16:57.436347 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 10:16:57.436347 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 10:16:57.436347 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 10:16:57.436347 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 10:16:57.436347 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 10:16:57.436347 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 10:16:57.436347 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:16:57.436347 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:16:57.436347 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 626/1244] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:16:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:16:57.436966 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c176f-0d81-4904-8b3b-b9e1bbdfbaea None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:16:57.552207 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c176f-0d81-4904-8b3b-b9e1bbdfbaea None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:16:57.586238 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34388af2-34b1-4a17-973e-ece81f61a0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:16:57.586879 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34388af2-34b1-4a17-973e-ece81f61a0b5 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:57.587197 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34388af2-34b1-4a17-973e-ece81f61a0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:57.587475 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34388af2-34b1-4a17-973e-ece81f61a0b5 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:57.588257 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34388af2-34b1-4a17-973e-ece81f61a0b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:16:57.588568 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34388af2-34b1-4a17-973e-ece81f61a0b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:16:57.590012 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34388af2-34b1-4a17-973e-ece81f61a0b5 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:16:57.602460 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 619/1245] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 10:16:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:16:57.609014 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8559793-792c-4f11-abb6-56b52aa83639 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:16:57.713380 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8559793-792c-4f11-abb6-56b52aa83639 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:16:57.747857 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a89040c-d0b2-46ba-8d8c-ae9edf4f9cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:16:57.748496 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89040c-d0b2-46ba-8d8c-ae9edf4f9cae None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:57.748770 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89040c-d0b2-46ba-8d8c-ae9edf4f9cae None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:57.749038 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89040c-d0b2-46ba-8d8c-ae9edf4f9cae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:57.749827 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a89040c-d0b2-46ba-8d8c-ae9edf4f9cae None admin] RBAC: Adding query filter params (role.id=3a2b4319820a488198dabee34420cdc7, scope.project.id=36b8b505012d4d0bac54845b6799e743, user.id=8743f5bb0be54d57a957617636b0b8fe) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:16:57.750123 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a89040c-d0b2-46ba-8d8c-ae9edf4f9cae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:16:57.751588 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a89040c-d0b2-46ba-8d8c-ae9edf4f9cae None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:16:57.758143 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 627/1246] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:16:57 2022] GET /identity/v3/role_assignments?user.id=8743f5bb0be54d57a957617636b0b8fe&scope.project.id=36b8b505012d4d0bac54845b6799e743&role.id=3a2b4319820a488198dabee34420cdc7 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:16:59.851520 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6e80a4-653a-4f10-b67b-45d8bdec87c9 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:59.852143 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6e80a4-653a-4f10-b67b-45d8bdec87c9 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:59.852143 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6e80a4-653a-4f10-b67b-45d8bdec87c9 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:59.853429 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 620/1247] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:16:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:16:59.858864 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c533c-79e3-4d7c-8524-f5a3914558d8 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:16:59.859316 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c533c-79e3-4d7c-8524-f5a3914558d8 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:16:59.859530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c533c-79e3-4d7c-8524-f5a3914558d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:16:59.867094 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:16:59.880175 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-399c533c-79e3-4d7c-8524-f5a3914558d8 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:16:59.881417 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-399c533c-79e3-4d7c-8524-f5a3914558d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:16:59.957719 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 628/1248] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:16:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:16:59.981249 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6e80a4-653a-4f10-b67b-45d8bdec87c9 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:16:59.999717 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:17:00.089458 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6e80a4-653a-4f10-b67b-45d8bdec87c9 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:00.127011 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0948bcbc-f3e4-4415-b193-37c4f3a98b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:00.127868 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0948bcbc-f3e4-4415-b193-37c4f3a98b06 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:00.128197 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0948bcbc-f3e4-4415-b193-37c4f3a98b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:00.128419 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0948bcbc-f3e4-4415-b193-37c4f3a98b06 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:00.139474 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0948bcbc-f3e4-4415-b193-37c4f3a98b06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:17:00.144465 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-0948bcbc-f3e4-4415-b193-37c4f3a98b06 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:17:00.144465 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:17:00.144465 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:17:00.144465 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:17:00.144465 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:17:00.144465 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:17:00.144465 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:17:00.144465 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:17:00.144465 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:17:00.144465 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:17:00.144465 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:17:00.144465 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:17:00.144465 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:17:00.144465 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:17:00.144465 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:17:00.144465 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:17:00.145793 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:17:00.145793 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:17:00.145793 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:17:00.145793 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:17:00.145793 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:17:00.145793 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:17:00.145793 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:17:00.145793 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:17:00.145793 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:17:00.145793 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:17:00.145793 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:17:00.145793 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:17:00.145793 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:17:00.145793 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:17:00.145793 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:17:00.145793 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:17:00.146700 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:17:00.146700 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:17:00.146700 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:17:00.146700 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:17:00.146700 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:17:00.146700 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:17:00.146700 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:17:00.146700 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:17:00.146700 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:17:00.146700 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:17:00.146700 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:17:00.146700 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:17:00.146700 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 621/1249] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 10:16:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:17:00.152351 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399c533c-79e3-4d7c-8524-f5a3914558d8 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:17:00.267402 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399c533c-79e3-4d7c-8524-f5a3914558d8 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:00.301094 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:17:00.306438 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da96c935-7295-4bbb-8ec6-fd2a793d49ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:00.307428 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96c935-7295-4bbb-8ec6-fd2a793d49ff None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:00.307861 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96c935-7295-4bbb-8ec6-fd2a793d49ff None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:00.308133 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96c935-7295-4bbb-8ec6-fd2a793d49ff None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:00.308828 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da96c935-7295-4bbb-8ec6-fd2a793d49ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:17:00.309125 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da96c935-7295-4bbb-8ec6-fd2a793d49ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:17:00.310490 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da96c935-7295-4bbb-8ec6-fd2a793d49ff None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:17:00.320093 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 629/1250] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:17:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:17:00.326662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0948bcbc-f3e4-4415-b193-37c4f3a98b06 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:17:00.411958 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:17:00.435989 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0948bcbc-f3e4-4415-b193-37c4f3a98b06 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:00.471924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b1b06a-67d9-48d5-b32b-75e77e922f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:00.472584 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1b06a-67d9-48d5-b32b-75e77e922f90 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:00.472904 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1b06a-67d9-48d5-b32b-75e77e922f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:00.473165 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1b06a-67d9-48d5-b32b-75e77e922f90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:00.487117 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b1b06a-67d9-48d5-b32b-75e77e922f90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:17:00.488666 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b1b06a-67d9-48d5-b32b-75e77e922f90 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:17:00.508110 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-a9b1b06a-67d9-48d5-b32b-75e77e922f90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:17:00.508110 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:17:00.508110 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:17:00.508110 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:17:00.508110 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:17:00.508110 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:17:00.508110 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:17:00.508110 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:17:00.508110 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:17:00.508110 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:17:00.508110 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:17:00.508110 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:17:00.508110 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:17:00.508110 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:17:00.508110 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:17:00.509195 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:17:00.509195 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:17:00.509195 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:17:00.509195 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:17:00.509195 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:17:00.509195 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:17:00.509195 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:17:00.509195 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:17:00.509195 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:17:00.509195 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:17:00.509195 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:17:00.509195 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:17:00.509195 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:17:00.509195 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:17:00.509195 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:17:00.509195 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:17:00.509195 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:17:00.510141 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:17:00.510141 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:17:00.510141 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:17:00.510141 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:17:00.510141 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 622/1251] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:17:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:17:00.520870 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da96c935-7295-4bbb-8ec6-fd2a793d49ff None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:17:00.631567 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da96c935-7295-4bbb-8ec6-fd2a793d49ff None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:00.665923 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111cf5c4-6a53-43fa-b6c1-403fcc05fe82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:00.666452 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111cf5c4-6a53-43fa-b6c1-403fcc05fe82 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:00.666729 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111cf5c4-6a53-43fa-b6c1-403fcc05fe82 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:00.667042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111cf5c4-6a53-43fa-b6c1-403fcc05fe82 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:00.667753 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111cf5c4-6a53-43fa-b6c1-403fcc05fe82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:17:00.668037 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111cf5c4-6a53-43fa-b6c1-403fcc05fe82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:17:00.669416 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111cf5c4-6a53-43fa-b6c1-403fcc05fe82 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:17:00.686687 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 630/1252] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:17:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:17:00.687984 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b1b06a-67d9-48d5-b32b-75e77e922f90 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:17:00.812042 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b1b06a-67d9-48d5-b32b-75e77e922f90 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:00.841407 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:17:00.849873 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03d2b98-efdf-43b1-b82b-35f042dce059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:00.850613 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d2b98-efdf-43b1-b82b-35f042dce059 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:00.850895 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d2b98-efdf-43b1-b82b-35f042dce059 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:00.851250 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d2b98-efdf-43b1-b82b-35f042dce059 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:00.859400 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03d2b98-efdf-43b1-b82b-35f042dce059 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:17:00.860906 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03d2b98-efdf-43b1-b82b-35f042dce059 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:17:00.873286 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-b03d2b98-efdf-43b1-b82b-35f042dce059 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 05 10:17:00.873286 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:17:00.873286 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:17:00.873286 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:17:00.873286 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:17:00.873286 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:17:00.873286 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:17:00.873286 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:17:00.873286 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:17:00.873286 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:17:00.873286 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:17:00.873286 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:17:00.873286 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 10:17:00.873286 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 10:17:00.873286 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 10:17:00.873286 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 10:17:00.874491 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:17:00.874491 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:17:00.874491 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 10:17:00.874491 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:17:00.874491 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 10:17:00.874491 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:17:00.874491 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:17:00.874491 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:17:00.874491 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:17:00.874491 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:17:00.874491 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:17:00.874491 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:17:00.874491 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:17:00.874491 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:17:00.874491 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:17:00.874491 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:17:00.874491 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:17:00.874491 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:17:00.875866 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:17:00.875866 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:17:00.875866 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 10:17:00.875866 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 10:17:00.875866 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 10:17:00.875866 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 10:17:00.875866 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 10:17:00.875866 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 10:17:00.875866 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 05 10:17:00.875866 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:17:00.875866 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 623/1253] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:17:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:17:00.881510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111cf5c4-6a53-43fa-b6c1-403fcc05fe82 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:17:00.947541 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:17:00.998724 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111cf5c4-6a53-43fa-b6c1-403fcc05fe82 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:01.037644 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f271468-8fe8-421a-85b7-ac2f468b5988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:01.038304 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f271468-8fe8-421a-85b7-ac2f468b5988 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:01.038585 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f271468-8fe8-421a-85b7-ac2f468b5988 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:01.038858 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f271468-8fe8-421a-85b7-ac2f468b5988 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:01.039704 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f271468-8fe8-421a-85b7-ac2f468b5988 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:17:01.040014 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f271468-8fe8-421a-85b7-ac2f468b5988 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:17:01.041460 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f271468-8fe8-421a-85b7-ac2f468b5988 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:17:01.061631 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 631/1254] 10.222.0.40 () {56 vars in 1143 bytes} [Fri Aug 5 10:17:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:17:01.066603 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03d2b98-efdf-43b1-b82b-35f042dce059 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:17:01.189132 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03d2b98-efdf-43b1-b82b-35f042dce059 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:01.230825 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c9e683-fd7e-402b-8d74-836c861b8293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:01.231618 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c9e683-fd7e-402b-8d74-836c861b8293 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:01.231860 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c9e683-fd7e-402b-8d74-836c861b8293 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:01.232312 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c9e683-fd7e-402b-8d74-836c861b8293 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:01.295604 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:17:01.310191 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c9e683-fd7e-402b-8d74-836c861b8293 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:17:01.311789 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c9e683-fd7e-402b-8d74-836c861b8293 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:17:01.436589 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 624/1255] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:17:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 05 10:17:01.443164 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f271468-8fe8-421a-85b7-ac2f468b5988 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:17:01.561184 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f271468-8fe8-421a-85b7-ac2f468b5988 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:01.576170 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:17:01.598924 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a24db7-25c1-4e23-b066-4566ac7aa24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:01.599565 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a24db7-25c1-4e23-b066-4566ac7aa24d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:01.599856 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a24db7-25c1-4e23-b066-4566ac7aa24d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:01.600108 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a24db7-25c1-4e23-b066-4566ac7aa24d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:01.611263 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a24db7-25c1-4e23-b066-4566ac7aa24d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:17:01.612890 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a24db7-25c1-4e23-b066-4566ac7aa24d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:17:01.629614 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-54a24db7-25c1-4e23-b066-4566ac7aa24d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:17:01.629614 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:17:01.629614 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:17:01.629614 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:17:01.629614 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:17:01.629614 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:17:01.629614 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:17:01.629614 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:17:01.629614 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:17:01.629614 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:17:01.629614 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:17:01.629614 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:17:01.629614 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:17:01.629614 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:17:01.629614 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:17:01.630821 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:17:01.630821 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:17:01.630821 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:17:01.630821 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:17:01.630821 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:17:01.630821 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:17:01.630821 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:17:01.630821 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:17:01.630821 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:17:01.630821 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:17:01.630821 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:17:01.630821 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:17:01.630821 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:17:01.630821 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:17:01.630821 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:17:01.630821 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:17:01.630821 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:17:01.631849 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:17:01.631849 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:17:01.631849 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:17:01.631849 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:17:01.631849 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 632/1256] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:17:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:17:01.636874 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c9e683-fd7e-402b-8d74-836c861b8293 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:17:01.733971 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:17:01.746774 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c9e683-fd7e-402b-8d74-836c861b8293 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:01.787515 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d19005-267f-4a32-8402-1a02854c8e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:01.788136 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d19005-267f-4a32-8402-1a02854c8e1f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:01.788468 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d19005-267f-4a32-8402-1a02854c8e1f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:01.788760 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d19005-267f-4a32-8402-1a02854c8e1f None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:01.789714 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d19005-267f-4a32-8402-1a02854c8e1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:17:01.790066 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d19005-267f-4a32-8402-1a02854c8e1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:17:01.791613 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d19005-267f-4a32-8402-1a02854c8e1f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:17:01.804674 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 625/1257] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:17:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:17:01.810582 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a24db7-25c1-4e23-b066-4566ac7aa24d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:17:01.931929 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a24db7-25c1-4e23-b066-4566ac7aa24d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:01.971274 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e56f98-6daf-4007-b408-2b1ef7cd5b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:01.972028 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e56f98-6daf-4007-b408-2b1ef7cd5b54 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:01.972304 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e56f98-6daf-4007-b408-2b1ef7cd5b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:01.972599 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e56f98-6daf-4007-b408-2b1ef7cd5b54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:01.980146 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e56f98-6daf-4007-b408-2b1ef7cd5b54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:17:01.981728 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e56f98-6daf-4007-b408-2b1ef7cd5b54 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:17:01.994275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-65e56f98-6daf-4007-b408-2b1ef7cd5b54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:17:01.994275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:17:01.994275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:17:01.994275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:17:01.994275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:17:01.994275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:17:01.994275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:17:01.994275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:17:01.994275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:17:01.994275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:17:01.994275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:17:01.994275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:17:01.994275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 10:17:01.994275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 10:17:01.994275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 10:17:01.995512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 10:17:01.995512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:17:01.995512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:17:01.995512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:17:01.995512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:17:01.995512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:17:01.995512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:17:01.995512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:17:01.995512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:17:01.995512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:17:01.995512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:17:01.995512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:17:01.995512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:17:01.995512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:17:01.995512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:17:01.995512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:17:01.995512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:17:01.996603 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 10:17:01.996603 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 10:17:01.996603 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 10:17:01.996603 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 10:17:01.996603 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 10:17:01.996603 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 10:17:01.996603 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:17:01.996603 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:17:01.996603 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 633/1258] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:17:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:17:02.001527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d19005-267f-4a32-8402-1a02854c8e1f None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:17:02.117407 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d19005-267f-4a32-8402-1a02854c8e1f None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:02.155139 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e151f-394a-4c59-85a5-1b58b55c768f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:02.155775 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e151f-394a-4c59-85a5-1b58b55c768f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:02.156044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e151f-394a-4c59-85a5-1b58b55c768f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:02.156333 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e151f-394a-4c59-85a5-1b58b55c768f None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:02.157162 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54e151f-394a-4c59-85a5-1b58b55c768f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:17:02.157349 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54e151f-394a-4c59-85a5-1b58b55c768f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:17:02.159918 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54e151f-394a-4c59-85a5-1b58b55c768f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:17:02.174759 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 626/1259] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 10:17:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:17:02.180597 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e56f98-6daf-4007-b408-2b1ef7cd5b54 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:17:02.210186 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:17:02.298138 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e56f98-6daf-4007-b408-2b1ef7cd5b54 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:02.334257 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894450e5-8e97-42b0-a2f3-0861ef93ec74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:02.334946 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894450e5-8e97-42b0-a2f3-0861ef93ec74 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:02.335192 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894450e5-8e97-42b0-a2f3-0861ef93ec74 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:02.335481 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894450e5-8e97-42b0-a2f3-0861ef93ec74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:02.336261 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894450e5-8e97-42b0-a2f3-0861ef93ec74 None admin] RBAC: Adding query filter params (role.id=daef89adcaeb48a382a0bcb03340ccbb, scope.project.id=36b8b505012d4d0bac54845b6799e743, user.id=8743f5bb0be54d57a957617636b0b8fe) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:17:02.336546 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894450e5-8e97-42b0-a2f3-0861ef93ec74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:17:02.339111 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894450e5-8e97-42b0-a2f3-0861ef93ec74 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:17:02.344242 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 634/1260] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:17:02 2022] GET /identity/v3/role_assignments?user.id=8743f5bb0be54d57a957617636b0b8fe&scope.project.id=36b8b505012d4d0bac54845b6799e743&role.id=daef89adcaeb48a382a0bcb03340ccbb => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:17:04.401288 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b7faf-1478-4852-819a-f019e89ee21f None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:04.401903 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b7faf-1478-4852-819a-f019e89ee21f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:04.401903 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b7faf-1478-4852-819a-f019e89ee21f None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:04.403270 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 627/1261] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:17:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:17:04.409652 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c0338b-a5e8-4b2e-8677-0d4d99adf71a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:04.410193 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c0338b-a5e8-4b2e-8677-0d4d99adf71a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:04.410193 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c0338b-a5e8-4b2e-8677-0d4d99adf71a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:04.430353 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-04c0338b-a5e8-4b2e-8677-0d4d99adf71a None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:17:04.431609 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-04c0338b-a5e8-4b2e-8677-0d4d99adf71a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:17:04.508130 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 635/1262] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:17:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:17:04.530701 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b7faf-1478-4852-819a-f019e89ee21f None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:17:04.543399 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:17:04.639668 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b7faf-1478-4852-819a-f019e89ee21f None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:04.675353 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe8a441-c62a-4a95-9769-9bdba7667d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:04.675930 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe8a441-c62a-4a95-9769-9bdba7667d44 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:04.676208 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe8a441-c62a-4a95-9769-9bdba7667d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:04.676480 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe8a441-c62a-4a95-9769-9bdba7667d44 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:04.686958 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe8a441-c62a-4a95-9769-9bdba7667d44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:17:04.691944 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-2fe8a441-c62a-4a95-9769-9bdba7667d44 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:17:04.691944 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:17:04.691944 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:17:04.691944 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:17:04.691944 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:17:04.691944 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:17:04.691944 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:17:04.691944 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:17:04.691944 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:17:04.691944 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:17:04.691944 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:17:04.691944 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:17:04.691944 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:17:04.691944 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:17:04.691944 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:17:04.691944 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:17:04.693053 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:17:04.693053 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:17:04.693053 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:17:04.693053 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:17:04.693053 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:17:04.693053 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:17:04.693053 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:17:04.693053 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:17:04.693053 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:17:04.693053 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:17:04.693053 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:17:04.693053 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:17:04.693053 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:17:04.693053 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:17:04.693053 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:17:04.693053 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:17:04.694034 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:17:04.694034 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:17:04.694034 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:17:04.694034 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:17:04.694034 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:17:04.694034 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:17:04.694034 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:17:04.694034 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:17:04.694034 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:17:04.694034 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:17:04.694034 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:17:04.694034 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:17:04.694034 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 628/1263] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 10:17:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:17:04.699403 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c0338b-a5e8-4b2e-8677-0d4d99adf71a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:17:04.723658 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:17:04.802992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c0338b-a5e8-4b2e-8677-0d4d99adf71a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:04.837352 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83f938e-929e-482a-8286-c2aa7d136240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:04.837983 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f938e-929e-482a-8286-c2aa7d136240 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:04.838250 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f938e-929e-482a-8286-c2aa7d136240 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:04.838522 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f938e-929e-482a-8286-c2aa7d136240 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:04.839263 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83f938e-929e-482a-8286-c2aa7d136240 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:17:04.839562 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83f938e-929e-482a-8286-c2aa7d136240 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:17:04.840935 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83f938e-929e-482a-8286-c2aa7d136240 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:17:04.849387 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 636/1264] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:17:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:17:04.855897 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe8a441-c62a-4a95-9769-9bdba7667d44 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:17:04.936272 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:17:04.972696 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe8a441-c62a-4a95-9769-9bdba7667d44 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:05.010221 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6461a656-0869-43d0-bce4-378c4376f57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:05.010856 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461a656-0869-43d0-bce4-378c4376f57b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:05.011172 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461a656-0869-43d0-bce4-378c4376f57b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:05.011435 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461a656-0869-43d0-bce4-378c4376f57b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:05.023034 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6461a656-0869-43d0-bce4-378c4376f57b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:17:05.024510 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6461a656-0869-43d0-bce4-378c4376f57b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:17:05.040419 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-6461a656-0869-43d0-bce4-378c4376f57b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:17:05.040419 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:17:05.040419 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:17:05.040419 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:17:05.040419 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:17:05.040419 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:17:05.040419 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:17:05.040419 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:17:05.040419 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:17:05.040419 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:17:05.040419 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:17:05.040419 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:17:05.040419 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:17:05.040419 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:17:05.040419 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:17:05.041440 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:17:05.041440 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:17:05.041440 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:17:05.041440 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:17:05.041440 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:17:05.041440 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:17:05.041440 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:17:05.041440 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:17:05.041440 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:17:05.041440 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:17:05.041440 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:17:05.041440 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:17:05.041440 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:17:05.041440 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:17:05.041440 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:17:05.041440 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:17:05.041440 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:17:05.043833 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:17:05.043833 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:17:05.043833 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:17:05.043833 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:17:05.043833 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 629/1265] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:17:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:17:05.047839 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83f938e-929e-482a-8286-c2aa7d136240 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:17:05.134613 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:17:05.165029 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83f938e-929e-482a-8286-c2aa7d136240 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:05.202144 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9634b96-9d49-4a2c-b1bb-a34a90ac37fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:05.202685 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9634b96-9d49-4a2c-b1bb-a34a90ac37fc None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:05.203002 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9634b96-9d49-4a2c-b1bb-a34a90ac37fc None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:05.203272 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9634b96-9d49-4a2c-b1bb-a34a90ac37fc None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:05.203966 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9634b96-9d49-4a2c-b1bb-a34a90ac37fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:17:05.204271 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9634b96-9d49-4a2c-b1bb-a34a90ac37fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:17:05.205650 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9634b96-9d49-4a2c-b1bb-a34a90ac37fc None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:17:05.217552 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 637/1266] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:17:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:17:05.224502 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6461a656-0869-43d0-bce4-378c4376f57b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:17:05.335175 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6461a656-0869-43d0-bce4-378c4376f57b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:05.358553 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:17:05.375315 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b173b9-476a-483e-9ba7-b8c1acdba865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:05.375948 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b173b9-476a-483e-9ba7-b8c1acdba865 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:05.376222 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b173b9-476a-483e-9ba7-b8c1acdba865 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:05.376491 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b173b9-476a-483e-9ba7-b8c1acdba865 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:05.384235 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b173b9-476a-483e-9ba7-b8c1acdba865 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:17:05.385664 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b173b9-476a-483e-9ba7-b8c1acdba865 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:17:05.396673 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-c0b173b9-476a-483e-9ba7-b8c1acdba865 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 05 10:17:05.396673 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:17:05.396673 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:17:05.396673 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:17:05.396673 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:17:05.396673 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:17:05.396673 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:17:05.396673 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:17:05.396673 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:17:05.396673 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:17:05.396673 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:17:05.396673 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:17:05.396673 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 10:17:05.396673 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 10:17:05.396673 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 10:17:05.396673 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 10:17:05.397729 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:17:05.397729 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:17:05.397729 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 10:17:05.397729 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:17:05.397729 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 10:17:05.397729 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:17:05.397729 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:17:05.397729 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:17:05.397729 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:17:05.397729 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:17:05.397729 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:17:05.397729 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:17:05.397729 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:17:05.397729 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:17:05.397729 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:17:05.397729 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:17:05.397729 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:17:05.397729 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:17:05.398859 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:17:05.398859 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:17:05.398859 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 10:17:05.398859 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 10:17:05.398859 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 10:17:05.398859 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 10:17:05.398859 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 10:17:05.398859 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 10:17:05.398859 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 05 10:17:05.398859 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:17:05.398859 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 630/1267] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:17:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:17:05.404022 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9634b96-9d49-4a2c-b1bb-a34a90ac37fc None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:17:05.511770 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9634b96-9d49-4a2c-b1bb-a34a90ac37fc None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:05.548660 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83775861-7246-4ec6-a2e9-da41b51c0bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:05.549500 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83775861-7246-4ec6-a2e9-da41b51c0bcd None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:05.549778 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83775861-7246-4ec6-a2e9-da41b51c0bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:05.550056 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83775861-7246-4ec6-a2e9-da41b51c0bcd None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:05.550838 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83775861-7246-4ec6-a2e9-da41b51c0bcd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:17:05.551182 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83775861-7246-4ec6-a2e9-da41b51c0bcd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:17:05.552627 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83775861-7246-4ec6-a2e9-da41b51c0bcd None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:17:05.561238 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:17:05.570280 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 638/1268] 10.222.0.40 () {56 vars in 1143 bytes} [Fri Aug 5 10:17:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:17:05.576978 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b173b9-476a-483e-9ba7-b8c1acdba865 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:17:05.689069 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b173b9-476a-483e-9ba7-b8c1acdba865 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:05.724549 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49523ed6-7c0f-4ac9-8c5e-bd2e26d03075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:05.725167 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49523ed6-7c0f-4ac9-8c5e-bd2e26d03075 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:05.725441 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49523ed6-7c0f-4ac9-8c5e-bd2e26d03075 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:05.725705 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49523ed6-7c0f-4ac9-8c5e-bd2e26d03075 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:05.737717 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49523ed6-7c0f-4ac9-8c5e-bd2e26d03075 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:17:05.739195 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49523ed6-7c0f-4ac9-8c5e-bd2e26d03075 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:17:05.753592 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-49523ed6-7c0f-4ac9-8c5e-bd2e26d03075 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:17:05.753592 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:17:05.753592 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:17:05.753592 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:17:05.753592 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:17:05.753592 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:17:05.753592 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:17:05.753592 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:17:05.753592 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:17:05.753592 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:17:05.753592 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:17:05.753592 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:17:05.753592 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:17:05.753592 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:17:05.753592 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:17:05.754580 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:17:05.754580 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:17:05.754580 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:17:05.754580 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:17:05.754580 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:17:05.754580 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:17:05.754580 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:17:05.754580 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:17:05.754580 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:17:05.754580 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:17:05.754580 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:17:05.754580 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:17:05.754580 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:17:05.754580 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:17:05.754580 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:17:05.754580 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:17:05.754580 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:17:05.755660 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:17:05.755660 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:17:05.755660 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:17:05.755660 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:17:05.755660 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 631/1269] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:17:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:17:05.761001 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83775861-7246-4ec6-a2e9-da41b51c0bcd None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:17:05.871922 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83775861-7246-4ec6-a2e9-da41b51c0bcd None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:05.917698 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76909337-e3ff-4135-8f0d-00ae63a395af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:05.918380 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76909337-e3ff-4135-8f0d-00ae63a395af None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:05.918743 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76909337-e3ff-4135-8f0d-00ae63a395af None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:05.919141 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76909337-e3ff-4135-8f0d-00ae63a395af None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:05.920094 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76909337-e3ff-4135-8f0d-00ae63a395af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:17:05.920505 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76909337-e3ff-4135-8f0d-00ae63a395af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:17:05.922393 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76909337-e3ff-4135-8f0d-00ae63a395af None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:17:05.938724 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 639/1270] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:17:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:17:05.944658 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49523ed6-7c0f-4ac9-8c5e-bd2e26d03075 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:17:05.982170 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:17:06.065258 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49523ed6-7c0f-4ac9-8c5e-bd2e26d03075 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:06.101333 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b1a2e-1f47-401e-86d5-080d91785b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:06.101966 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b1a2e-1f47-401e-86d5-080d91785b6c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:06.102245 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b1a2e-1f47-401e-86d5-080d91785b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:06.102518 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b1a2e-1f47-401e-86d5-080d91785b6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:06.109514 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3b1a2e-1f47-401e-86d5-080d91785b6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:17:06.110995 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3b1a2e-1f47-401e-86d5-080d91785b6c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:17:06.123292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-8d3b1a2e-1f47-401e-86d5-080d91785b6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:17:06.123292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:17:06.123292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:17:06.123292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:17:06.123292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:17:06.123292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:17:06.123292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:17:06.123292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:17:06.123292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:17:06.123292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:17:06.123292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:17:06.123292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:17:06.123292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 10:17:06.123292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 10:17:06.123292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 10:17:06.124244 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 10:17:06.124244 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:17:06.124244 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:17:06.124244 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:17:06.124244 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:17:06.124244 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:17:06.124244 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:17:06.124244 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:17:06.124244 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:17:06.124244 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:17:06.124244 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:17:06.124244 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:17:06.124244 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:17:06.124244 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:17:06.124244 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:17:06.124244 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:17:06.124244 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:17:06.125528 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 10:17:06.125528 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 10:17:06.125528 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 10:17:06.125528 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 10:17:06.125528 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 10:17:06.125528 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 10:17:06.125528 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:17:06.125528 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:17:06.125528 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 632/1271] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:17:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:17:06.130611 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76909337-e3ff-4135-8f0d-00ae63a395af None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:17:06.192747 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:17:06.238288 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76909337-e3ff-4135-8f0d-00ae63a395af None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:06.273896 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e22923-9eb8-42ed-8c89-27913f379bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:06.274515 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e22923-9eb8-42ed-8c89-27913f379bde None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:06.274800 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e22923-9eb8-42ed-8c89-27913f379bde None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:06.275109 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e22923-9eb8-42ed-8c89-27913f379bde None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:06.275843 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e22923-9eb8-42ed-8c89-27913f379bde None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:17:06.276148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e22923-9eb8-42ed-8c89-27913f379bde None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:17:06.277591 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e22923-9eb8-42ed-8c89-27913f379bde None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:17:06.290249 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 640/1272] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 10:17:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:17:06.296693 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b1a2e-1f47-401e-86d5-080d91785b6c None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:17:06.392486 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:17:06.402258 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b1a2e-1f47-401e-86d5-080d91785b6c None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:06.443188 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2452c12-654b-4f2a-8526-3b2ac301e136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:06.443831 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2452c12-654b-4f2a-8526-3b2ac301e136 None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:06.444111 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2452c12-654b-4f2a-8526-3b2ac301e136 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:06.444385 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2452c12-654b-4f2a-8526-3b2ac301e136 None admin] PATH_INFO: `/v3/projects/36b8b505012d4d0bac54845b6799e743/users/8743f5bb0be54d57a957617636b0b8fe/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:06.446420 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2452c12-654b-4f2a-8526-3b2ac301e136 None admin] RBAC: Authorizing `identity:create_grant(project_id=36b8b505012d4d0bac54845b6799e743, user_id=8743f5bb0be54d57a957617636b0b8fe, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:17:06.447904 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2452c12-654b-4f2a-8526-3b2ac301e136 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:17:06.467672 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 633/1273] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 10:17:06 2022] PUT /identity/v3/projects/36b8b505012d4d0bac54845b6799e743/users/8743f5bb0be54d57a957617636b0b8fe/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:17:08.417085 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b0004-9227-4bd4-a87b-85fd1253efdd None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:08.417631 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b0004-9227-4bd4-a87b-85fd1253efdd None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:08.417631 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b0004-9227-4bd4-a87b-85fd1253efdd None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:08.418884 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 641/1274] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:17:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:17:08.425613 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b72830-ff28-4e13-9d94-afbd08eee118 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:08.426350 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b72830-ff28-4e13-9d94-afbd08eee118 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:08.426534 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b72830-ff28-4e13-9d94-afbd08eee118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:08.450791 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-12b72830-ff28-4e13-9d94-afbd08eee118 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:17:08.452003 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-12b72830-ff28-4e13-9d94-afbd08eee118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:17:08.523600 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 634/1275] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:17:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:17:08.557065 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b0004-9227-4bd4-a87b-85fd1253efdd None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:17:08.662646 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b0004-9227-4bd4-a87b-85fd1253efdd None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:08.679816 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:17:08.705239 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7213fd5b-eb10-445f-a114-dad853121242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:08.705870 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7213fd5b-eb10-445f-a114-dad853121242 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:08.706144 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7213fd5b-eb10-445f-a114-dad853121242 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:08.706407 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7213fd5b-eb10-445f-a114-dad853121242 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:08.717143 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7213fd5b-eb10-445f-a114-dad853121242 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 10:17:08.722100 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-7213fd5b-eb10-445f-a114-dad853121242 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:17:08.722100 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:17:08.722100 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:17:08.722100 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:17:08.722100 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:17:08.722100 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:17:08.722100 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:17:08.722100 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:17:08.722100 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:17:08.722100 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:17:08.722100 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:17:08.722100 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:17:08.722100 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 10:17:08.722100 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 10:17:08.722100 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 10:17:08.722100 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 10:17:08.723221 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 10:17:08.723221 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 10:17:08.723221 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 10:17:08.723221 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 10:17:08.723221 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:17:08.723221 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:17:08.723221 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:17:08.723221 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:17:08.723221 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:17:08.723221 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:17:08.723221 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:17:08.723221 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:17:08.723221 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:17:08.723221 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:17:08.723221 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:17:08.723221 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:17:08.724168 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:17:08.724168 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:17:08.724168 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:17:08.724168 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:17:08.724168 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 10:17:08.724168 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 10:17:08.724168 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 10:17:08.724168 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 10:17:08.724168 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 10:17:08.724168 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 10:17:08.724168 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 10:17:08.724168 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:17:08.724168 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 642/1276] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 10:17:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:17:08.729419 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b72830-ff28-4e13-9d94-afbd08eee118 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:17:08.812181 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:17:08.832023 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b72830-ff28-4e13-9d94-afbd08eee118 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:08.867607 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4167962d-ec24-4ffd-991d-b787c45ef186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:08.868211 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4167962d-ec24-4ffd-991d-b787c45ef186 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:08.868489 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4167962d-ec24-4ffd-991d-b787c45ef186 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:08.868754 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4167962d-ec24-4ffd-991d-b787c45ef186 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:08.869461 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4167962d-ec24-4ffd-991d-b787c45ef186 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:17:08.869760 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4167962d-ec24-4ffd-991d-b787c45ef186 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:17:08.871137 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4167962d-ec24-4ffd-991d-b787c45ef186 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:17:08.880761 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 635/1277] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 10:17:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:17:08.887692 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7213fd5b-eb10-445f-a114-dad853121242 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:17:09.000768 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7213fd5b-eb10-445f-a114-dad853121242 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:09.037189 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e193b4-38a1-4203-bb76-a418e4fbc9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:09.037827 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e193b4-38a1-4203-bb76-a418e4fbc9ef None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:09.038102 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e193b4-38a1-4203-bb76-a418e4fbc9ef None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:09.038354 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e193b4-38a1-4203-bb76-a418e4fbc9ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:09.049038 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e193b4-38a1-4203-bb76-a418e4fbc9ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:17:09.050578 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e193b4-38a1-4203-bb76-a418e4fbc9ef None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:17:09.066261 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-75e193b4-38a1-4203-bb76-a418e4fbc9ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:17:09.066261 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:17:09.066261 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:17:09.066261 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:17:09.066261 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:17:09.066261 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:17:09.066261 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:17:09.066261 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:17:09.066261 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:17:09.066261 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:17:09.066261 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:17:09.066261 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:17:09.066261 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:17:09.066261 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:17:09.066261 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:17:09.067319 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:17:09.067319 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:17:09.067319 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:17:09.067319 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:17:09.067319 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:17:09.067319 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:17:09.067319 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:17:09.067319 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:17:09.067319 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:17:09.067319 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:17:09.067319 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:17:09.067319 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:17:09.067319 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:17:09.067319 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:17:09.067319 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:17:09.067319 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:17:09.067319 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:17:09.069010 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:17:09.069010 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:17:09.069010 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:17:09.069010 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:17:09.069010 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 643/1278] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:17:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:17:09.076194 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4167962d-ec24-4ffd-991d-b787c45ef186 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:17:09.183537 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4167962d-ec24-4ffd-991d-b787c45ef186 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:09.219144 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3433723-35b2-4ec1-a3cb-ac36f17bd364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:09.219753 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3433723-35b2-4ec1-a3cb-ac36f17bd364 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:09.220053 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3433723-35b2-4ec1-a3cb-ac36f17bd364 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:09.220548 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3433723-35b2-4ec1-a3cb-ac36f17bd364 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:09.221230 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3433723-35b2-4ec1-a3cb-ac36f17bd364 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:17:09.221531 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3433723-35b2-4ec1-a3cb-ac36f17bd364 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:17:09.222953 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3433723-35b2-4ec1-a3cb-ac36f17bd364 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:17:09.235350 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 636/1279] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:17:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:17:09.244099 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e193b4-38a1-4203-bb76-a418e4fbc9ef None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:17:09.280317 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:17:09.353425 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e193b4-38a1-4203-bb76-a418e4fbc9ef None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:09.395559 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5a4310-9dbc-4269-9919-7f5acfa716ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:09.396243 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a4310-9dbc-4269-9919-7f5acfa716ca None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:09.396520 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a4310-9dbc-4269-9919-7f5acfa716ca None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:09.396790 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a4310-9dbc-4269-9919-7f5acfa716ca None admin] PATH_INFO: `/v3/users/placement` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:09.404720 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5a4310-9dbc-4269-9919-7f5acfa716ca None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:17:09.406183 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5a4310-9dbc-4269-9919-7f5acfa716ca None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:17:09.417609 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-6e5a4310-9dbc-4269-9919-7f5acfa716ca None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 05 10:17:09.417609 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:17:09.417609 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:17:09.417609 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:17:09.417609 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:17:09.417609 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:17:09.417609 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:17:09.417609 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:17:09.417609 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:17:09.417609 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:17:09.417609 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:17:09.417609 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:17:09.417609 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 10:17:09.417609 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 10:17:09.417609 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 10:17:09.417609 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 10:17:09.419184 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:17:09.419184 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:17:09.419184 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 10:17:09.419184 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:17:09.419184 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 10:17:09.419184 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 10:17:09.419184 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:17:09.419184 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:17:09.419184 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:17:09.419184 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:17:09.419184 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:17:09.419184 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:17:09.419184 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:17:09.419184 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:17:09.419184 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:17:09.419184 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:17:09.419184 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:17:09.419184 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:17:09.420193 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:17:09.420193 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:17:09.420193 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 10:17:09.420193 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 10:17:09.420193 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 10:17:09.420193 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 10:17:09.420193 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 10:17:09.420193 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 10:17:09.420193 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 05 10:17:09.420193 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:17:09.420193 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 644/1280] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:17:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 10:17:09.425065 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3433723-35b2-4ec1-a3cb-ac36f17bd364 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:17:09.431004 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:17:09.541127 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3433723-35b2-4ec1-a3cb-ac36f17bd364 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:09.582832 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8763d6-6b16-4691-a9b0-c607f445cb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:09.583488 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8763d6-6b16-4691-a9b0-c607f445cb63 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:09.583765 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8763d6-6b16-4691-a9b0-c607f445cb63 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:09.584029 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8763d6-6b16-4691-a9b0-c607f445cb63 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:09.584818 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8763d6-6b16-4691-a9b0-c607f445cb63 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:17:09.585119 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8763d6-6b16-4691-a9b0-c607f445cb63 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:17:09.586569 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8763d6-6b16-4691-a9b0-c607f445cb63 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:17:09.606746 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 637/1281] 10.222.0.40 () {56 vars in 1143 bytes} [Fri Aug 5 10:17:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:17:09.613304 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5a4310-9dbc-4269-9919-7f5acfa716ca None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:17:09.731095 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:17:09.741934 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5a4310-9dbc-4269-9919-7f5acfa716ca None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:09.777483 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcab4254-8520-49e7-ab62-09b6a6d6d5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:09.778107 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab4254-8520-49e7-ab62-09b6a6d6d5c7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:09.778378 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab4254-8520-49e7-ab62-09b6a6d6d5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:09.778646 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab4254-8520-49e7-ab62-09b6a6d6d5c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:09.861577 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcab4254-8520-49e7-ab62-09b6a6d6d5c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:17:09.863075 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcab4254-8520-49e7-ab62-09b6a6d6d5c7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:17:09.983754 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:17:09.996990 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 645/1282] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 10:17:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 10:17:10.006125 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8763d6-6b16-4691-a9b0-c607f445cb63 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:17:10.078167 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:17:10.126884 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8763d6-6b16-4691-a9b0-c607f445cb63 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:10.172539 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f67a7d-c9bd-40ec-a3a2-cd687a11e6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:10.173459 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f67a7d-c9bd-40ec-a3a2-cd687a11e6c3 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:10.173819 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f67a7d-c9bd-40ec-a3a2-cd687a11e6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:10.174322 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f67a7d-c9bd-40ec-a3a2-cd687a11e6c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:10.185169 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f67a7d-c9bd-40ec-a3a2-cd687a11e6c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:17:10.186623 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f67a7d-c9bd-40ec-a3a2-cd687a11e6c3 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:17:10.201997 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-e1f67a7d-c9bd-40ec-a3a2-cd687a11e6c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:17:10.201997 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:17:10.201997 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:17:10.201997 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:17:10.201997 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:17:10.201997 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:17:10.201997 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:17:10.201997 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:17:10.201997 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:17:10.201997 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:17:10.201997 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:17:10.201997 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:17:10.201997 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 10:17:10.201997 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 10:17:10.201997 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 10:17:10.203085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 10:17:10.203085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:17:10.203085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:17:10.203085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:17:10.203085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:17:10.203085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:17:10.203085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:17:10.203085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:17:10.203085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:17:10.203085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:17:10.203085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:17:10.203085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:17:10.203085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:17:10.203085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:17:10.203085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:17:10.203085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:17:10.203085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:17:10.204165 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 10:17:10.204165 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 10:17:10.204165 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 10:17:10.204165 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:17:10.204165 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 638/1283] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:17:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:17:10.209690 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcab4254-8520-49e7-ab62-09b6a6d6d5c7 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:17:10.327662 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcab4254-8520-49e7-ab62-09b6a6d6d5c7 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:10.368326 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327d3a1a-5ea0-4b48-8969-926967e900b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:10.369193 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d3a1a-5ea0-4b48-8969-926967e900b9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:10.369548 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d3a1a-5ea0-4b48-8969-926967e900b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:10.369922 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d3a1a-5ea0-4b48-8969-926967e900b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:10.371007 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327d3a1a-5ea0-4b48-8969-926967e900b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:17:10.371420 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327d3a1a-5ea0-4b48-8969-926967e900b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:17:10.373528 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327d3a1a-5ea0-4b48-8969-926967e900b9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:17:10.386808 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 646/1284] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 10:17:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:17:10.392759 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f67a7d-c9bd-40ec-a3a2-cd687a11e6c3 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:17:10.501806 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f67a7d-c9bd-40ec-a3a2-cd687a11e6c3 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:10.516074 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:17:10.544183 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63a953b-7dd8-4a70-aea5-895ff5526a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:10.545042 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a953b-7dd8-4a70-aea5-895ff5526a7d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:10.545457 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a953b-7dd8-4a70-aea5-895ff5526a7d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:10.545848 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a953b-7dd8-4a70-aea5-895ff5526a7d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:10.558103 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63a953b-7dd8-4a70-aea5-895ff5526a7d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:17:10.560479 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63a953b-7dd8-4a70-aea5-895ff5526a7d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:17:10.574999 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-a63a953b-7dd8-4a70-aea5-895ff5526a7d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:17:10.574999 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:17:10.574999 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:17:10.574999 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:17:10.574999 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:17:10.574999 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:17:10.574999 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:17:10.574999 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:17:10.574999 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:17:10.574999 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:17:10.574999 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:17:10.574999 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:17:10.574999 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 10:17:10.574999 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 10:17:10.574999 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 10:17:10.576032 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 10:17:10.576032 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:17:10.576032 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:17:10.576032 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:17:10.576032 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:17:10.576032 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:17:10.576032 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:17:10.576032 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:17:10.576032 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 05 10:17:10.576032 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:17:10.576032 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:17:10.576032 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:17:10.576032 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:17:10.576032 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:17:10.576032 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:17:10.576032 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:17:10.576032 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:17:10.576963 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 10:17:10.576963 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 10:17:10.576963 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 10:17:10.576963 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 10:17:10.576963 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 10:17:10.576963 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 10:17:10.576963 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 10:17:10.576963 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 05 10:17:10.576963 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 639/1285] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 10:17:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:17:10.582636 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327d3a1a-5ea0-4b48-8969-926967e900b9 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:17:10.636258 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:17:10.688571 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327d3a1a-5ea0-4b48-8969-926967e900b9 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:10.726685 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ee7ed-7fa8-4eec-9d77-12862090b092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:10.727336 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ee7ed-7fa8-4eec-9d77-12862090b092 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:10.727614 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ee7ed-7fa8-4eec-9d77-12862090b092 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:10.727885 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ee7ed-7fa8-4eec-9d77-12862090b092 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:10.728624 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85ee7ed-7fa8-4eec-9d77-12862090b092 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:17:10.728967 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85ee7ed-7fa8-4eec-9d77-12862090b092 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:17:10.730403 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85ee7ed-7fa8-4eec-9d77-12862090b092 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:17:10.742780 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 647/1286] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 10:17:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:17:10.749162 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63a953b-7dd8-4a70-aea5-895ff5526a7d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:17:10.855375 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63a953b-7dd8-4a70-aea5-895ff5526a7d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:10.895936 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0156c76e-69f4-4829-868a-bedb9191b7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:10.896538 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156c76e-69f4-4829-868a-bedb9191b7aa None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:10.896822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156c76e-69f4-4829-868a-bedb9191b7aa None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:10.897098 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156c76e-69f4-4829-868a-bedb9191b7aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:10.897898 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0156c76e-69f4-4829-868a-bedb9191b7aa None admin] RBAC: Adding query filter params (role.id=daef89adcaeb48a382a0bcb03340ccbb, scope.project.id=36b8b505012d4d0bac54845b6799e743, user.id=8743f5bb0be54d57a957617636b0b8fe) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:17:10.898203 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0156c76e-69f4-4829-868a-bedb9191b7aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:17:10.899799 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0156c76e-69f4-4829-868a-bedb9191b7aa None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:17:10.906024 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 640/1287] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 10:17:10 2022] GET /identity/v3/role_assignments?user.id=8743f5bb0be54d57a957617636b0b8fe&scope.project.id=36b8b505012d4d0bac54845b6799e743&role.id=daef89adcaeb48a382a0bcb03340ccbb => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:17:12.912637 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac493e2-e9ba-4914-9288-0ce945ddc613 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:12.913218 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac493e2-e9ba-4914-9288-0ce945ddc613 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:12.913218 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac493e2-e9ba-4914-9288-0ce945ddc613 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:12.915215 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 648/1288] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:17:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:17:12.922618 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4917815-d229-47ec-b059-e8cbd80c61f8 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:12.923068 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4917815-d229-47ec-b059-e8cbd80c61f8 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:12.923187 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4917815-d229-47ec-b059-e8cbd80c61f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:12.943077 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a4917815-d229-47ec-b059-e8cbd80c61f8 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:17:12.944255 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a4917815-d229-47ec-b059-e8cbd80c61f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:17:12.974940 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:17:13.018282 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 641/1289] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:17:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:17:13.040279 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac493e2-e9ba-4914-9288-0ce945ddc613 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:17:13.159462 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac493e2-e9ba-4914-9288-0ce945ddc613 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:13.171379 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:17:13.198049 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ed08e8-ba84-42f4-9187-e8f277bcf4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:13.198678 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed08e8-ba84-42f4-9187-e8f277bcf4a6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:13.198958 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed08e8-ba84-42f4-9187-e8f277bcf4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:13.199255 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed08e8-ba84-42f4-9187-e8f277bcf4a6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:13.199873 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ed08e8-ba84-42f4-9187-e8f277bcf4a6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:17:13.201237 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ed08e8-ba84-42f4-9187-e8f277bcf4a6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:17:13.210928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-e8ed08e8-ba84-42f4-9187-e8f277bcf4a6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 10:17:13.210928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:17:13.210928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 10:17:13.210928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 10:17:13.210928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 10:17:13.210928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 10:17:13.210928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 10:17:13.210928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:17:13.210928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 10:17:13.210928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:17:13.210928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 10:17:13.210928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:17:13.210928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:17:13.210928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:17:13.210928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:17:13.210928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:17:13.210928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:17:13.212118 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:17:13.212118 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:17:13.212118 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:17:13.212118 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:17:13.212118 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:17:13.212118 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:17:13.212118 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 10:17:13.212118 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 10:17:13.212118 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 10:17:13.212118 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 10:17:13.212118 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:17:13.212118 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:17:13.212118 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:17:13.212118 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:17:13.212118 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:17:13.212118 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:17:13.213730 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:17:13.213730 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:17:13.213730 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:17:13.213730 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:17:13.213730 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:17:13.213730 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:17:13.213730 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:17:13.213730 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:17:13.213730 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:17:13.213730 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:17:13.213730 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 10:17:13.213730 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:17:13.213730 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 10:17:13.213730 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:17:13.213730 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 649/1290] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Aug 5 10:17:13 2022] GET /identity/v3/services/placement => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:17:13.221496 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4917815-d229-47ec-b059-e8cbd80c61f8 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:17:13.329165 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4917815-d229-47ec-b059-e8cbd80c61f8 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:13.365310 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26120170-dba2-4529-92a6-c0fd22fc733b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:13.365926 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26120170-dba2-4529-92a6-c0fd22fc733b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:13.366205 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26120170-dba2-4529-92a6-c0fd22fc733b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:13.366479 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26120170-dba2-4529-92a6-c0fd22fc733b None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:13.367188 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26120170-dba2-4529-92a6-c0fd22fc733b None admin] RBAC: Adding query filter params (name=placement) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:17:13.367493 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26120170-dba2-4529-92a6-c0fd22fc733b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:17:13.368876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26120170-dba2-4529-92a6-c0fd22fc733b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:17:13.373904 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 642/1291] 10.222.0.40 () {56 vars in 1116 bytes} [Fri Aug 5 10:17:13 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:17:13.381250 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ed08e8-ba84-42f4-9187-e8f277bcf4a6 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:17:13.495848 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ed08e8-ba84-42f4-9187-e8f277bcf4a6 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:13.530369 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00450923-af85-4ae9-8026-444486510829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:13.530972 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00450923-af85-4ae9-8026-444486510829 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:13.531269 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00450923-af85-4ae9-8026-444486510829 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:13.531537 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00450923-af85-4ae9-8026-444486510829 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:13.532200 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00450923-af85-4ae9-8026-444486510829 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:17:13.532499 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00450923-af85-4ae9-8026-444486510829 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:17:13.534351 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00450923-af85-4ae9-8026-444486510829 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:17:13.539089 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 650/1292] 10.222.0.40 () {56 vars in 1116 bytes} [Fri Aug 5 10:17:13 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:17:15.531005 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3be008-113d-4e5f-afd7-f2562459bbc1 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:15.531676 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3be008-113d-4e5f-afd7-f2562459bbc1 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:15.531676 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3be008-113d-4e5f-afd7-f2562459bbc1 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:15.532955 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 643/1293] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:17:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:17:15.539603 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd8e59-2ec1-4513-84f0-81146b4d9f50 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:15.540355 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd8e59-2ec1-4513-84f0-81146b4d9f50 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:15.540496 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd8e59-2ec1-4513-84f0-81146b4d9f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:15.562750 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-bbcd8e59-2ec1-4513-84f0-81146b4d9f50 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:17:15.563988 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bbcd8e59-2ec1-4513-84f0-81146b4d9f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:17:15.595252 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:17:15.634885 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 651/1294] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:17:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:17:15.657273 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3be008-113d-4e5f-afd7-f2562459bbc1 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:17:15.693368 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:17:15.765847 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3be008-113d-4e5f-afd7-f2562459bbc1 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:15.807414 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fda6e5-1639-481c-81fd-c5952ec3f1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:15.807946 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fda6e5-1639-481c-81fd-c5952ec3f1f3 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:15.808223 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fda6e5-1639-481c-81fd-c5952ec3f1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:15.808490 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fda6e5-1639-481c-81fd-c5952ec3f1f3 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:15.809229 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fda6e5-1639-481c-81fd-c5952ec3f1f3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:17:15.810657 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fda6e5-1639-481c-81fd-c5952ec3f1f3 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:17:15.834217 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 644/1295] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 5 10:17:15 2022] POST /identity/v3/services => generated 246 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:17:17.833965 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf7daf-0024-4fe8-b39e-c5d2a7d195f1 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:17.835527 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf7daf-0024-4fe8-b39e-c5d2a7d195f1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:17.835989 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf7daf-0024-4fe8-b39e-c5d2a7d195f1 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:17.838250 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 652/1296] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:17:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:17:17.843787 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4bec3e-f8b8-4b86-921f-fbd5acef6358 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:17.844216 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4bec3e-f8b8-4b86-921f-fbd5acef6358 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:17.844311 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4bec3e-f8b8-4b86-921f-fbd5acef6358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:17.864079 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-6e4bec3e-f8b8-4b86-921f-fbd5acef6358 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:17:17.865265 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6e4bec3e-f8b8-4b86-921f-fbd5acef6358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:17:17.948444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 645/1297] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:17:17 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:17:17.972286 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf7daf-0024-4fe8-b39e-c5d2a7d195f1 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:17:18.085276 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf7daf-0024-4fe8-b39e-c5d2a7d195f1 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:18.119978 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eab671-4494-4d12-b0ad-4ce9a4dfa22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:18.120610 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eab671-4494-4d12-b0ad-4ce9a4dfa22d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:18.120884 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eab671-4494-4d12-b0ad-4ce9a4dfa22d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:18.121148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eab671-4494-4d12-b0ad-4ce9a4dfa22d None admin] PATH_INFO: `/v3/services/placement` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:18.121760 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eab671-4494-4d12-b0ad-4ce9a4dfa22d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:17:18.123160 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eab671-4494-4d12-b0ad-4ce9a4dfa22d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:17:18.136594 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-05eab671-4494-4d12-b0ad-4ce9a4dfa22d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 10:17:18.136594 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:17:18.136594 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 10:17:18.136594 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 10:17:18.136594 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 10:17:18.136594 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 10:17:18.136594 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 10:17:18.136594 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:17:18.136594 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 10:17:18.136594 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:17:18.136594 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 10:17:18.136594 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:17:18.136594 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:17:18.136594 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:17:18.136594 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:17:18.136594 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:17:18.136594 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:17:18.137784 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:17:18.137784 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:17:18.137784 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:17:18.137784 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:17:18.137784 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:17:18.137784 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:17:18.137784 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 10:17:18.137784 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 10:17:18.137784 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 10:17:18.137784 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 10:17:18.137784 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:17:18.137784 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:17:18.137784 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:17:18.137784 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:17:18.137784 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:17:18.137784 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:17:18.138727 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:17:18.138727 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:17:18.138727 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:17:18.138727 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:17:18.138727 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:17:18.138727 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:17:18.138727 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:17:18.138727 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:17:18.138727 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:17:18.138727 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:17:18.138727 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 10:17:18.138727 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:17:18.138727 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 10:17:18.138727 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:17:18.138727 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 653/1298] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Aug 5 10:17:17 2022] GET /identity/v3/services/placement => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:17:18.147378 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4bec3e-f8b8-4b86-921f-fbd5acef6358 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:17:18.168702 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:17:18.251349 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4bec3e-f8b8-4b86-921f-fbd5acef6358 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:18.287352 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50c2e1c-4990-483c-9d28-9986af92f57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:18.287957 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c2e1c-4990-483c-9d28-9986af92f57d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:18.288228 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c2e1c-4990-483c-9d28-9986af92f57d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:18.288495 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c2e1c-4990-483c-9d28-9986af92f57d None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:18.289171 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50c2e1c-4990-483c-9d28-9986af92f57d None admin] RBAC: Adding query filter params (name=placement) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:17:18.289478 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50c2e1c-4990-483c-9d28-9986af92f57d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:17:18.290845 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50c2e1c-4990-483c-9d28-9986af92f57d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:17:18.296137 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 646/1299] 10.222.0.40 () {56 vars in 1116 bytes} [Fri Aug 5 10:17:18 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:17:18.302665 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eab671-4494-4d12-b0ad-4ce9a4dfa22d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:17:18.336363 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:17:18.427103 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eab671-4494-4d12-b0ad-4ce9a4dfa22d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:18.462706 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3fe82-9c4e-4245-9dcb-6b115796bbb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:18.463350 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3fe82-9c4e-4245-9dcb-6b115796bbb7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:18.463629 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3fe82-9c4e-4245-9dcb-6b115796bbb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:18.463897 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3fe82-9c4e-4245-9dcb-6b115796bbb7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:18.464640 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d3fe82-9c4e-4245-9dcb-6b115796bbb7 None admin] RBAC: Adding query filter params (interface=public, service_id=527edb43ea1740a98b638496afbb4da5, region_id=RegionOne) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:17:18.464945 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d3fe82-9c4e-4245-9dcb-6b115796bbb7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:17:18.466330 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d3fe82-9c4e-4245-9dcb-6b115796bbb7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:17:18.471625 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 654/1300] 10.222.0.40 () {56 vars in 1251 bytes} [Fri Aug 5 10:17:18 2022] GET /identity/v3/endpoints?service_id=527edb43ea1740a98b638496afbb4da5&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:17:18.477786 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50c2e1c-4990-483c-9d28-9986af92f57d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:17:18.565603 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:17:18.584614 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50c2e1c-4990-483c-9d28-9986af92f57d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:18.624117 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90806ab1-90d9-49bd-b095-4a021c7ea236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:18.624897 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90806ab1-90d9-49bd-b095-4a021c7ea236 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:18.625272 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90806ab1-90d9-49bd-b095-4a021c7ea236 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:18.625639 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90806ab1-90d9-49bd-b095-4a021c7ea236 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:18.626668 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90806ab1-90d9-49bd-b095-4a021c7ea236 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:17:18.628505 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90806ab1-90d9-49bd-b095-4a021c7ea236 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:17:18.635844 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 647/1301] 10.222.0.40 () {56 vars in 1087 bytes} [Fri Aug 5 10:17:18 2022] GET /identity/v3/services => generated 2191 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:17:20.605653 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9acfd0-cb47-4422-86d8-4161783c7e26 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:20.606237 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9acfd0-cb47-4422-86d8-4161783c7e26 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:20.606237 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9acfd0-cb47-4422-86d8-4161783c7e26 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:20.607602 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 655/1302] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:17:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:17:20.612969 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca3b26-3f96-4f7c-95f5-ee3579e39fe1 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:20.613404 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca3b26-3f96-4f7c-95f5-ee3579e39fe1 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:20.613496 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca3b26-3f96-4f7c-95f5-ee3579e39fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:20.633333 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-7cca3b26-3f96-4f7c-95f5-ee3579e39fe1 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:17:20.634511 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7cca3b26-3f96-4f7c-95f5-ee3579e39fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:17:20.709051 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 648/1303] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:17:20 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 10:17:20.731546 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9acfd0-cb47-4422-86d8-4161783c7e26 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:17:20.836452 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:17:20.844255 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9acfd0-cb47-4422-86d8-4161783c7e26 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:20.884226 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f7f27-816c-4231-b405-00a9fce65501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:20.884783 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f7f27-816c-4231-b405-00a9fce65501 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:20.885089 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f7f27-816c-4231-b405-00a9fce65501 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:20.885384 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f7f27-816c-4231-b405-00a9fce65501 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:20.886063 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5f7f27-816c-4231-b405-00a9fce65501 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:17:20.887638 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5f7f27-816c-4231-b405-00a9fce65501 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:17:20.898401 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-9a5f7f27-816c-4231-b405-00a9fce65501 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 10:17:20.898401 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:17:20.898401 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 10:17:20.898401 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 10:17:20.898401 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 10:17:20.898401 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 10:17:20.898401 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 10:17:20.898401 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:17:20.898401 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 10:17:20.898401 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:17:20.898401 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 10:17:20.898401 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:17:20.898401 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 10:17:20.898401 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 10:17:20.898401 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 10:17:20.898401 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 10:17:20.898401 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 10:17:20.899690 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 10:17:20.899690 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 10:17:20.899690 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 10:17:20.899690 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 10:17:20.899690 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 10:17:20.899690 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 10:17:20.899690 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 10:17:20.899690 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 10:17:20.899690 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 10:17:20.899690 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 10:17:20.899690 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 10:17:20.899690 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 10:17:20.899690 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 10:17:20.899690 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 10:17:20.899690 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 10:17:20.899690 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 10:17:20.900750 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 10:17:20.900750 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Aug 05 10:17:20.900750 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 10:17:20.900750 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Aug 05 10:17:20.900750 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 10:17:20.900750 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 10:17:20.900750 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 10:17:20.900750 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Aug 05 10:17:20.900750 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 10:17:20.900750 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Aug 05 10:17:20.900750 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 10:17:20.900750 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 10:17:20.900750 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 10:17:20.900750 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Aug 05 10:17:20.900750 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 656/1304] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Aug 5 10:17:20 2022] GET /identity/v3/services/placement => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 10:17:20.906668 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cca3b26-3f96-4f7c-95f5-ee3579e39fe1 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:17:20.992330 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:17:21.016578 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cca3b26-3f96-4f7c-95f5-ee3579e39fe1 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:21.052376 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94de09c-6dce-49d3-91e5-69a5b19ca7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:21.052995 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94de09c-6dce-49d3-91e5-69a5b19ca7eb None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:21.053279 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94de09c-6dce-49d3-91e5-69a5b19ca7eb None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:21.053550 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94de09c-6dce-49d3-91e5-69a5b19ca7eb None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:21.054228 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94de09c-6dce-49d3-91e5-69a5b19ca7eb None admin] RBAC: Adding query filter params (name=placement) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:17:21.054537 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94de09c-6dce-49d3-91e5-69a5b19ca7eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:17:21.055948 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94de09c-6dce-49d3-91e5-69a5b19ca7eb None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:17:21.061272 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 649/1305] 10.222.0.40 () {56 vars in 1116 bytes} [Fri Aug 5 10:17:20 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:17:21.068021 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f7f27-816c-4231-b405-00a9fce65501 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:17:21.177115 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f7f27-816c-4231-b405-00a9fce65501 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:17:21.212539 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac6d40f-a16e-4539-b515-1e1e1a429e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:17:21.213153 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac6d40f-a16e-4539-b515-1e1e1a429e22 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:17:21.213432 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac6d40f-a16e-4539-b515-1e1e1a429e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:17:21.213704 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac6d40f-a16e-4539-b515-1e1e1a429e22 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:17:21.214408 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac6d40f-a16e-4539-b515-1e1e1a429e22 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:17:21.215817 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac6d40f-a16e-4539-b515-1e1e1a429e22 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:17:21.240572 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 657/1306] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:17:21 2022] POST /identity/v3/endpoints => generated 331 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:17:31.058024 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=30 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:17:31.235269 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:18:24.485735 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70503f61-2d65-493e-958e-03a921c57c4f None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:18:24.486357 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70503f61-2d65-493e-958e-03a921c57c4f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:18:24.486357 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70503f61-2d65-493e-958e-03a921c57c4f None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:18:24.491811 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 650/1307] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:18:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:18:24.493213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9839e1e9-116d-43fb-a2ac-c22dbc57c7ef None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:18:24.493637 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9839e1e9-116d-43fb-a2ac-c22dbc57c7ef None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:18:24.493836 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9839e1e9-116d-43fb-a2ac-c22dbc57c7ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:18:24.570173 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9839e1e9-116d-43fb-a2ac-c22dbc57c7ef None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:18:24.571418 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9839e1e9-116d-43fb-a2ac-c22dbc57c7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:18:24.694092 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 658/1308] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:18:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:18:24.764114 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46764ad1-cf11-4c5a-92ed-60514118be0b None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:18:24.764405 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46764ad1-cf11-4c5a-92ed-60514118be0b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:18:24.764681 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46764ad1-cf11-4c5a-92ed-60514118be0b None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:18:24.766250 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 651/1309] 10.222.0.40 () {54 vars in 965 bytes} [Fri Aug 5 10:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:18:24.775500 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eda7a7-ae5f-4140-95aa-a7574337c3ac None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:18:24.775975 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eda7a7-ae5f-4140-95aa-a7574337c3ac None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:18:24.776369 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eda7a7-ae5f-4140-95aa-a7574337c3ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:18:24.899890 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a1eda7a7-ae5f-4140-95aa-a7574337c3ac None None] MFA Rules not processed for user `a7deb94069a449269cb03406232174d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:18:24.902729 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a1eda7a7-ae5f-4140-95aa-a7574337c3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:18:24.927066 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:18:24.992455 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 659/1310] 10.222.0.40 () {58 vars in 1062 bytes} [Fri Aug 5 10:18:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:18:25.001958 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46764ad1-cf11-4c5a-92ed-60514118be0b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:18:25.003968 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-46764ad1-cf11-4c5a-92ed-60514118be0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:18:25.126937 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46764ad1-cf11-4c5a-92ed-60514118be0b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:18:25.206304 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcd8cf6-9240-4fae-91ca-5d44b9c1eefe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:18:25.206938 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd8cf6-9240-4fae-91ca-5d44b9c1eefe service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:18:25.207277 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd8cf6-9240-4fae-91ca-5d44b9c1eefe service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:18:25.207548 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd8cf6-9240-4fae-91ca-5d44b9c1eefe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:18:25.209319 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9fcd8cf6-9240-4fae-91ca-5d44b9c1eefe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:18:25.268515 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:18:25.326700 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcd8cf6-9240-4fae-91ca-5d44b9c1eefe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:18:25.328273 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcd8cf6-9240-4fae-91ca-5d44b9c1eefe service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:18:25.348663 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 652/1311] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:18:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:18:27.628814 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346c73e-e29d-45cd-a7ba-aceef1cb891d None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:18:27.629339 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346c73e-e29d-45cd-a7ba-aceef1cb891d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:18:27.629339 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346c73e-e29d-45cd-a7ba-aceef1cb891d None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:18:27.630776 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 660/1312] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:18:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 10:18:27.636089 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513de58b-452e-4423-9dc3-dc2648f7de63 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:18:27.636526 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513de58b-452e-4423-9dc3-dc2648f7de63 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:18:27.636604 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513de58b-452e-4423-9dc3-dc2648f7de63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:18:27.724348 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-513de58b-452e-4423-9dc3-dc2648f7de63 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:18:27.725557 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-513de58b-452e-4423-9dc3-dc2648f7de63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:18:27.827468 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 653/1313] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:18:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:18:27.865782 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346c73e-e29d-45cd-a7ba-aceef1cb891d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:18:27.867372 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8346c73e-e29d-45cd-a7ba-aceef1cb891d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:18:27.989829 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346c73e-e29d-45cd-a7ba-aceef1cb891d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:18:28.071219 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:18:28.073928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6194afab-5fb4-4ffa-8c89-e5a3b499f12d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:18:28.074758 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6194afab-5fb4-4ffa-8c89-e5a3b499f12d service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:18:28.075225 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6194afab-5fb4-4ffa-8c89-e5a3b499f12d service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:18:28.075925 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6194afab-5fb4-4ffa-8c89-e5a3b499f12d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:18:28.077576 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6194afab-5fb4-4ffa-8c89-e5a3b499f12d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:18:28.191581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6194afab-5fb4-4ffa-8c89-e5a3b499f12d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:18:28.193120 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6194afab-5fb4-4ffa-8c89-e5a3b499f12d service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:18:28.215901 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 661/1314] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:18:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:18:30.509388 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7a3fa-0f38-4139-be02-7a67d588363e None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:18:30.510281 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7a3fa-0f38-4139-be02-7a67d588363e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:18:30.510281 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7a3fa-0f38-4139-be02-7a67d588363e None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:18:30.512653 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 654/1315] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:18:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:18:30.518862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c6979-544e-42cd-9976-50192613de4a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:18:30.519515 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c6979-544e-42cd-9976-50192613de4a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:18:30.519676 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c6979-544e-42cd-9976-50192613de4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:18:30.587189 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ee3c6979-544e-42cd-9976-50192613de4a None None] MFA Rules not processed for user `f720a2ce258149afacb65b2c45fa2173`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:18:30.588376 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ee3c6979-544e-42cd-9976-50192613de4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:18:30.647181 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:18:30.696497 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 662/1316] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:18:30 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:18:30.741437 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee7a3fa-0f38-4139-be02-7a67d588363e None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:18:30.743018 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6ee7a3fa-0f38-4139-be02-7a67d588363e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:18:30.753828 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:18:30.861662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee7a3fa-0f38-4139-be02-7a67d588363e None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:18:30.948486 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba734b-d054-4b7a-9996-e44b5a5c5666 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:18:30.949117 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba734b-d054-4b7a-9996-e44b5a5c5666 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:18:30.949397 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba734b-d054-4b7a-9996-e44b5a5c5666 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:18:30.949670 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba734b-d054-4b7a-9996-e44b5a5c5666 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:18:30.951344 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a7ba734b-d054-4b7a-9996-e44b5a5c5666 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:18:31.021185 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:18:31.071402 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ba734b-d054-4b7a-9996-e44b5a5c5666 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:18:31.072846 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ba734b-d054-4b7a-9996-e44b5a5c5666 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:18:31.093020 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 655/1317] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:18:30 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:18:33.677015 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3fa1e4-bd47-4a39-8415-09044554afa9 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:18:33.677613 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3fa1e4-bd47-4a39-8415-09044554afa9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:18:33.677613 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3fa1e4-bd47-4a39-8415-09044554afa9 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:18:33.679082 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 663/1318] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:18:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:18:33.684570 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077507fa-9e54-4855-96b5-5b9cbf32642c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:18:33.684989 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077507fa-9e54-4855-96b5-5b9cbf32642c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:18:33.685115 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077507fa-9e54-4855-96b5-5b9cbf32642c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:18:33.761665 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-077507fa-9e54-4855-96b5-5b9cbf32642c None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:18:33.762880 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-077507fa-9e54-4855-96b5-5b9cbf32642c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:18:33.868676 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 656/1319] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:18:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:18:33.911819 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3fa1e4-bd47-4a39-8415-09044554afa9 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:18:33.914368 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ed3fa1e4-bd47-4a39-8415-09044554afa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:18:34.034453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3fa1e4-bd47-4a39-8415-09044554afa9 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:18:34.109983 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e21fc-f380-437d-8e53-a99c4201418e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:18:34.110606 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e21fc-f380-437d-8e53-a99c4201418e service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:18:34.110882 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e21fc-f380-437d-8e53-a99c4201418e service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:18:34.111194 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e21fc-f380-437d-8e53-a99c4201418e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:18:34.112862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-153e21fc-f380-437d-8e53-a99c4201418e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:18:34.120538 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:18:34.229935 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153e21fc-f380-437d-8e53-a99c4201418e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:18:34.231427 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153e21fc-f380-437d-8e53-a99c4201418e service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:18:34.250340 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 664/1320] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:18:33 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:18:36.921265 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72ebf9-286c-4452-b444-d3851343e6a1 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:18:36.921887 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72ebf9-286c-4452-b444-d3851343e6a1 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:18:36.921887 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72ebf9-286c-4452-b444-d3851343e6a1 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:18:36.923370 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 657/1321] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:18:36.928903 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ffbc07-0543-4cc1-b0c8-f6a249090f87 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:18:36.929350 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ffbc07-0543-4cc1-b0c8-f6a249090f87 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:18:36.929458 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ffbc07-0543-4cc1-b0c8-f6a249090f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:18:37.001027 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f5ffbc07-0543-4cc1-b0c8-f6a249090f87 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:18:37.002243 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f5ffbc07-0543-4cc1-b0c8-f6a249090f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:18:37.071624 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:18:37.115479 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 665/1322] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:18:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:18:37.168269 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6604fa-32d5-4d9f-bd7c-fa35d9d4acbb None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:18:37.168510 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6604fa-32d5-4d9f-bd7c-fa35d9d4acbb None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:18:37.168804 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6604fa-32d5-4d9f-bd7c-fa35d9d4acbb None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:18:37.170254 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 658/1323] 10.222.0.40 () {54 vars in 965 bytes} [Fri Aug 5 10:18:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:18:37.177171 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3ab5a-a7d7-4f8e-bc04-7ab75a285deb None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:18:37.177410 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3ab5a-a7d7-4f8e-bc04-7ab75a285deb None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:18:37.177647 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3ab5a-a7d7-4f8e-bc04-7ab75a285deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:18:37.270926 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f2b3ab5a-a7d7-4f8e-bc04-7ab75a285deb None None] MFA Rules not processed for user `a7deb94069a449269cb03406232174d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:18:37.272168 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f2b3ab5a-a7d7-4f8e-bc04-7ab75a285deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:18:37.353966 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 666/1324] 10.222.0.40 () {58 vars in 1062 bytes} [Fri Aug 5 10:18:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:18:37.363511 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6604fa-32d5-4d9f-bd7c-fa35d9d4acbb None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:18:37.365077 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9b6604fa-32d5-4d9f-bd7c-fa35d9d4acbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:18:37.384861 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:18:37.474273 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6604fa-32d5-4d9f-bd7c-fa35d9d4acbb None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:18:37.555365 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f1a1db-0148-4adb-9ebc-f6f836855d18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:18:37.555985 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f1a1db-0148-4adb-9ebc-f6f836855d18 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:18:37.556248 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f1a1db-0148-4adb-9ebc-f6f836855d18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:18:37.556517 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f1a1db-0148-4adb-9ebc-f6f836855d18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:18:37.558129 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-43f1a1db-0148-4adb-9ebc-f6f836855d18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:18:37.632728 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:18:37.673636 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f1a1db-0148-4adb-9ebc-f6f836855d18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:18:37.675120 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f1a1db-0148-4adb-9ebc-f6f836855d18 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:18:37.693380 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 659/1325] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:18:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:18:40.028332 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab57d59-4bff-42ef-aab8-6cb6977335ef None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:18:40.028933 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab57d59-4bff-42ef-aab8-6cb6977335ef None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:18:40.028933 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab57d59-4bff-42ef-aab8-6cb6977335ef None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:18:40.030227 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 667/1326] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:18:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:18:40.036141 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b0e26-795f-4ff7-83f2-76a5e4fb52ad None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:18:40.036571 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b0e26-795f-4ff7-83f2-76a5e4fb52ad None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:18:40.036678 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b0e26-795f-4ff7-83f2-76a5e4fb52ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:18:40.114291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-3e8b0e26-795f-4ff7-83f2-76a5e4fb52ad None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:18:40.115556 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3e8b0e26-795f-4ff7-83f2-76a5e4fb52ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:18:40.220417 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 660/1327] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:18:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:18:40.257064 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab57d59-4bff-42ef-aab8-6cb6977335ef None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:18:40.258599 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2ab57d59-4bff-42ef-aab8-6cb6977335ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:18:40.326050 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:18:40.371207 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab57d59-4bff-42ef-aab8-6cb6977335ef None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:18:40.447150 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a7043-14ee-4f79-8736-d94ad272778b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:18:40.447768 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a7043-14ee-4f79-8736-d94ad272778b service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:18:40.448044 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a7043-14ee-4f79-8736-d94ad272778b service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:18:40.448310 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a7043-14ee-4f79-8736-d94ad272778b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:18:40.449913 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3b6a7043-14ee-4f79-8736-d94ad272778b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:18:40.567732 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:18:40.570536 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6a7043-14ee-4f79-8736-d94ad272778b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:18:40.572076 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6a7043-14ee-4f79-8736-d94ad272778b service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:18:40.590694 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 668/1328] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:18:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:18:42.621813 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a73da-01eb-4418-94ed-3f84cf611552 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:18:42.622378 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a73da-01eb-4418-94ed-3f84cf611552 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:18:42.622378 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a73da-01eb-4418-94ed-3f84cf611552 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:18:42.623761 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 661/1329] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:18:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 10:18:42.629503 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5d540-024d-4836-a60f-9d16e4cc937c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:18:42.629930 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5d540-024d-4836-a60f-9d16e4cc937c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:18:42.630075 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5d540-024d-4836-a60f-9d16e4cc937c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:18:42.720871 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d8c5d540-024d-4836-a60f-9d16e4cc937c None None] MFA Rules not processed for user `f720a2ce258149afacb65b2c45fa2173`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:18:42.722276 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d8c5d540-024d-4836-a60f-9d16e4cc937c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:18:42.832100 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 669/1330] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:18:42 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:18:42.874853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463a73da-01eb-4418-94ed-3f84cf611552 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:18:42.876451 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-463a73da-01eb-4418-94ed-3f84cf611552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:18:42.915017 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:18:42.994786 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463a73da-01eb-4418-94ed-3f84cf611552 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:18:43.076589 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c31ac49-c5df-454c-80f4-bb602b78cb03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:18:43.077214 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31ac49-c5df-454c-80f4-bb602b78cb03 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:18:43.077493 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31ac49-c5df-454c-80f4-bb602b78cb03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:18:43.077756 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31ac49-c5df-454c-80f4-bb602b78cb03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:18:43.079417 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0c31ac49-c5df-454c-80f4-bb602b78cb03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:18:43.168542 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:18:43.209106 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c31ac49-c5df-454c-80f4-bb602b78cb03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:18:43.210641 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c31ac49-c5df-454c-80f4-bb602b78cb03 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:18:43.238480 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 662/1331] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:18:42 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:18:45.494743 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85611c77-d927-41d1-afad-f4fb0fb5fa1d None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:18:45.495313 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85611c77-d927-41d1-afad-f4fb0fb5fa1d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:18:45.495313 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85611c77-d927-41d1-afad-f4fb0fb5fa1d None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:18:45.497443 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 670/1332] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:18:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:18:45.506341 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6609064-dfd8-43f3-bf0a-f1ae1bbd8e43 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:18:45.507112 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6609064-dfd8-43f3-bf0a-f1ae1bbd8e43 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:18:45.507324 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6609064-dfd8-43f3-bf0a-f1ae1bbd8e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:18:45.580024 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c6609064-dfd8-43f3-bf0a-f1ae1bbd8e43 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:18:45.581307 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c6609064-dfd8-43f3-bf0a-f1ae1bbd8e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:18:45.684631 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 663/1333] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:18:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:18:45.729169 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85611c77-d927-41d1-afad-f4fb0fb5fa1d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:18:45.730736 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-85611c77-d927-41d1-afad-f4fb0fb5fa1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:18:45.804528 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:18:45.840327 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85611c77-d927-41d1-afad-f4fb0fb5fa1d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:18:45.926693 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09be209-c8bc-4f76-b030-3c528060d93f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:18:45.927711 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09be209-c8bc-4f76-b030-3c528060d93f service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:18:45.928163 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09be209-c8bc-4f76-b030-3c528060d93f service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:18:45.928606 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09be209-c8bc-4f76-b030-3c528060d93f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:18:45.931276 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c09be209-c8bc-4f76-b030-3c528060d93f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:18:46.043189 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09be209-c8bc-4f76-b030-3c528060d93f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:18:46.044666 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09be209-c8bc-4f76-b030-3c528060d93f service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:18:46.056604 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:18:46.065723 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 671/1334] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:18:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:18:48.139242 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf71a71-50a6-4a06-ab8b-56d4f181bb12 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:18:48.140271 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf71a71-50a6-4a06-ab8b-56d4f181bb12 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:18:48.140271 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf71a71-50a6-4a06-ab8b-56d4f181bb12 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:18:48.142315 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 664/1335] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:18:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:18:48.147758 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9bf08d-3d89-4ce2-910e-38304deda9ad None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:18:48.148208 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9bf08d-3d89-4ce2-910e-38304deda9ad None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:18:48.148313 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9bf08d-3d89-4ce2-910e-38304deda9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:18:48.227242 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-fe9bf08d-3d89-4ce2-910e-38304deda9ad None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:18:48.228464 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fe9bf08d-3d89-4ce2-910e-38304deda9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:18:48.335455 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 672/1336] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:18:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:18:48.379675 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf71a71-50a6-4a06-ab8b-56d4f181bb12 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:18:48.382265 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-dcf71a71-50a6-4a06-ab8b-56d4f181bb12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:18:48.431212 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:18:48.501228 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf71a71-50a6-4a06-ab8b-56d4f181bb12 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:18:48.578551 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74225896-7ef6-4898-825f-b3d8a0df1b38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:18:48.579205 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74225896-7ef6-4898-825f-b3d8a0df1b38 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:18:48.579483 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74225896-7ef6-4898-825f-b3d8a0df1b38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:18:48.579746 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74225896-7ef6-4898-825f-b3d8a0df1b38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:18:48.581677 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-74225896-7ef6-4898-825f-b3d8a0df1b38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:18:48.675372 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:18:48.702474 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74225896-7ef6-4898-825f-b3d8a0df1b38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:18:48.703894 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74225896-7ef6-4898-825f-b3d8a0df1b38 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:18:48.722247 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 665/1337] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:18:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:18:51.111451 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b63aa-5024-4791-81bc-95711758d8bc None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:18:51.112068 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b63aa-5024-4791-81bc-95711758d8bc None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:18:51.112068 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b63aa-5024-4791-81bc-95711758d8bc None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:18:51.113396 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 673/1338] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:18:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:18:51.119824 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd0640-a2fd-4243-aa84-0d8107d644d6 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:18:51.120597 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd0640-a2fd-4243-aa84-0d8107d644d6 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:18:51.120749 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd0640-a2fd-4243-aa84-0d8107d644d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:18:51.215946 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-1dcd0640-a2fd-4243-aa84-0d8107d644d6 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:18:51.217170 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1dcd0640-a2fd-4243-aa84-0d8107d644d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:18:51.331788 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 666/1339] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:18:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:18:51.368386 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b63aa-5024-4791-81bc-95711758d8bc None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:18:51.369930 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ce6b63aa-5024-4791-81bc-95711758d8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:18:51.464413 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:18:51.495313 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b63aa-5024-4791-81bc-95711758d8bc None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:18:51.577914 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a12bec-2953-4b2d-9b17-72d5bae55b40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:18:51.578649 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a12bec-2953-4b2d-9b17-72d5bae55b40 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:18:51.578943 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a12bec-2953-4b2d-9b17-72d5bae55b40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:18:51.579228 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a12bec-2953-4b2d-9b17-72d5bae55b40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:18:51.581048 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-17a12bec-2953-4b2d-9b17-72d5bae55b40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:18:51.704161 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a12bec-2953-4b2d-9b17-72d5bae55b40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:18:51.705810 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a12bec-2953-4b2d-9b17-72d5bae55b40 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:18:51.731300 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 674/1340] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:18:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:18:55.672867 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ab72c-0101-4de1-8675-902c1964d5c3 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:18:55.673453 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ab72c-0101-4de1-8675-902c1964d5c3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:18:55.673453 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ab72c-0101-4de1-8675-902c1964d5c3 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:18:55.674825 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 667/1341] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:18:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:18:55.680087 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144b3dd3-c577-415e-8140-43c62c688afb None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:18:55.680532 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144b3dd3-c577-415e-8140-43c62c688afb None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:18:55.680626 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144b3dd3-c577-415e-8140-43c62c688afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:18:55.686332 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:18:55.753383 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-144b3dd3-c577-415e-8140-43c62c688afb None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:18:55.754604 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-144b3dd3-c577-415e-8140-43c62c688afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:18:55.857571 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 675/1342] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:18:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:18:55.897931 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ab72c-0101-4de1-8675-902c1964d5c3 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:18:55.899505 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-324ab72c-0101-4de1-8675-902c1964d5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:18:55.951523 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:18:56.014777 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ab72c-0101-4de1-8675-902c1964d5c3 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:18:56.091236 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d8af2a-4d93-4876-90af-1ccc313ae3f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:18:56.091883 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d8af2a-4d93-4876-90af-1ccc313ae3f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:18:56.092166 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d8af2a-4d93-4876-90af-1ccc313ae3f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:18:56.092459 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d8af2a-4d93-4876-90af-1ccc313ae3f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:18:56.094069 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-25d8af2a-4d93-4876-90af-1ccc313ae3f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:18:56.198181 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:18:56.214769 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d8af2a-4d93-4876-90af-1ccc313ae3f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:18:56.216264 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d8af2a-4d93-4876-90af-1ccc313ae3f7 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:18:56.235210 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 668/1343] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:18:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:18:59.191614 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602362e3-92d2-41f8-b8d3-69f80ccdf50a None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:18:59.192134 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602362e3-92d2-41f8-b8d3-69f80ccdf50a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:18:59.192134 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602362e3-92d2-41f8-b8d3-69f80ccdf50a None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:18:59.193465 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 676/1344] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:18:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:18:59.199095 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b7031-0e03-4657-8c48-ff0da4c3c8d6 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:18:59.199493 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b7031-0e03-4657-8c48-ff0da4c3c8d6 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:18:59.199642 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b7031-0e03-4657-8c48-ff0da4c3c8d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:18:59.271154 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-534b7031-0e03-4657-8c48-ff0da4c3c8d6 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:18:59.272375 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-534b7031-0e03-4657-8c48-ff0da4c3c8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:18:59.369131 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 669/1345] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:18:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:18:59.407115 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602362e3-92d2-41f8-b8d3-69f80ccdf50a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:18:59.408709 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-602362e3-92d2-41f8-b8d3-69f80ccdf50a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:18:59.498248 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:18:59.516737 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602362e3-92d2-41f8-b8d3-69f80ccdf50a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:18:59.597975 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b2a66-c64d-4ebd-b7e9-1806a9758af5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:18:59.598607 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b2a66-c64d-4ebd-b7e9-1806a9758af5 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:18:59.598887 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b2a66-c64d-4ebd-b7e9-1806a9758af5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:18:59.599168 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b2a66-c64d-4ebd-b7e9-1806a9758af5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:18:59.600822 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e58b2a66-c64d-4ebd-b7e9-1806a9758af5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:18:59.716666 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58b2a66-c64d-4ebd-b7e9-1806a9758af5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:18:59.718123 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58b2a66-c64d-4ebd-b7e9-1806a9758af5 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:18:59.736803 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 677/1346] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:18:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:19:03.832930 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3851d5c-5aef-4f05-a231-0037fd14f2cf None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:03.833404 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3851d5c-5aef-4f05-a231-0037fd14f2cf None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:03.833482 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3851d5c-5aef-4f05-a231-0037fd14f2cf None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:03.834851 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 670/1347] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:19:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:19:03.840579 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27ac98-f62b-4819-9f7f-bb5d5b591efa None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:03.841115 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27ac98-f62b-4819-9f7f-bb5d5b591efa None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:03.841115 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27ac98-f62b-4819-9f7f-bb5d5b591efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:03.858555 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:19:03.918156 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-5d27ac98-f62b-4819-9f7f-bb5d5b591efa None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:19:03.919489 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5d27ac98-f62b-4819-9f7f-bb5d5b591efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:04.019345 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 678/1348] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:19:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:19:04.058340 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3851d5c-5aef-4f05-a231-0037fd14f2cf None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:19:04.059976 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d3851d5c-5aef-4f05-a231-0037fd14f2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:04.120866 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:19:04.174121 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3851d5c-5aef-4f05-a231-0037fd14f2cf None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:19:04.248922 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e011642e-bfb8-4aa0-9fe4-db240dcb1c0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:19:04.249585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e011642e-bfb8-4aa0-9fe4-db240dcb1c0b service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:04.249823 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e011642e-bfb8-4aa0-9fe4-db240dcb1c0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:04.250120 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e011642e-bfb8-4aa0-9fe4-db240dcb1c0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:04.251907 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e011642e-bfb8-4aa0-9fe4-db240dcb1c0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:04.370808 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:19:04.374222 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e011642e-bfb8-4aa0-9fe4-db240dcb1c0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:19:04.376093 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e011642e-bfb8-4aa0-9fe4-db240dcb1c0b service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:19:04.394319 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 671/1349] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:19:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:19:09.193953 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e3971-b47f-4891-8bd3-0785bf73ac21 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:09.194468 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e3971-b47f-4891-8bd3-0785bf73ac21 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:09.194548 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e3971-b47f-4891-8bd3-0785bf73ac21 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:09.196299 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 679/1350] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 5 10:19:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:19:09.201449 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5877ed8-2a06-40a1-8695-3fa4a97beef7 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:09.201880 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5877ed8-2a06-40a1-8695-3fa4a97beef7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:09.202071 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5877ed8-2a06-40a1-8695-3fa4a97beef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:09.311104 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a5877ed8-2a06-40a1-8695-3fa4a97beef7 None None] MFA Rules not processed for user `8743f5bb0be54d57a957617636b0b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:19:09.312632 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a5877ed8-2a06-40a1-8695-3fa4a97beef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:09.463430 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 672/1351] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 5 10:19:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:19:13.567185 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a003c9-f170-41c1-9f86-37fc1e8a1bed None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:13.567743 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a003c9-f170-41c1-9f86-37fc1e8a1bed None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:13.567743 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a003c9-f170-41c1-9f86-37fc1e8a1bed None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:13.569261 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 680/1352] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 5 10:19:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:19:13.574486 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8050f8-833d-4846-b724-7a029474534a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:13.575523 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8050f8-833d-4846-b724-7a029474534a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:13.575997 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8050f8-833d-4846-b724-7a029474534a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:13.638317 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:19:13.666673 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-2e8050f8-833d-4846-b724-7a029474534a None None] MFA Rules not processed for user `8743f5bb0be54d57a957617636b0b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:19:13.668185 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2e8050f8-833d-4846-b724-7a029474534a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:13.799683 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 673/1353] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 5 10:19:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:19:14.017816 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=59 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:19:15.232781 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a59d1-4753-4dc0-b5b0-99e31277bf83 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:15.233418 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a59d1-4753-4dc0-b5b0-99e31277bf83 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:15.233418 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a59d1-4753-4dc0-b5b0-99e31277bf83 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:15.235160 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 681/1354] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 5 10:19:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:19:15.240530 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f4133-1c45-4792-9642-0064cf46fd1e None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:15.241005 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f4133-1c45-4792-9642-0064cf46fd1e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:15.241101 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f4133-1c45-4792-9642-0064cf46fd1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:15.321855 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-984f4133-1c45-4792-9642-0064cf46fd1e None None] MFA Rules not processed for user `8743f5bb0be54d57a957617636b0b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:19:15.323121 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-984f4133-1c45-4792-9642-0064cf46fd1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:15.369010 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:19:15.439634 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 674/1355] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 5 10:19:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:19:18.931507 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e642caa5-33fe-40d2-a813-80840dafeae0 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:18.932518 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e642caa5-33fe-40d2-a813-80840dafeae0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:18.932998 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e642caa5-33fe-40d2-a813-80840dafeae0 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:18.939014 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 682/1356] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:19:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:19:18.941741 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5481b5a-25cc-4b5b-a22b-f0e97cff46ea None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:18.942604 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5481b5a-25cc-4b5b-a22b-f0e97cff46ea None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:18.943133 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5481b5a-25cc-4b5b-a22b-f0e97cff46ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:19.022970 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-e5481b5a-25cc-4b5b-a22b-f0e97cff46ea None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:19:19.024963 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e5481b5a-25cc-4b5b-a22b-f0e97cff46ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:19.127270 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 675/1357] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:19:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:19:19.181569 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a5cb8-0470-46ca-8ae4-45af3fdd6247 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:19.183253 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a5cb8-0470-46ca-8ae4-45af3fdd6247 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:19.183944 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a5cb8-0470-46ca-8ae4-45af3fdd6247 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:19.185841 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 683/1358] 10.222.0.40 () {54 vars in 962 bytes} [Fri Aug 5 10:19:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:19:19.194129 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cfc9bf-0e05-4b6b-8c7f-1d5d1e67ffe2 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:19.195058 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cfc9bf-0e05-4b6b-8c7f-1d5d1e67ffe2 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:19.195644 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cfc9bf-0e05-4b6b-8c7f-1d5d1e67ffe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:19.216292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:19:19.282875 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-12cfc9bf-0e05-4b6b-8c7f-1d5d1e67ffe2 None None] MFA Rules not processed for user `b938f1f41db94b0eafe95bc8eeaae175`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:19:19.284137 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-12cfc9bf-0e05-4b6b-8c7f-1d5d1e67ffe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:19.412072 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 676/1359] 10.222.0.40 () {58 vars in 1059 bytes} [Fri Aug 5 10:19:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:19:19.420590 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a5cb8-0470-46ca-8ae4-45af3fdd6247 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:19:19.422322 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0d5a5cb8-0470-46ca-8ae4-45af3fdd6247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:19.576901 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a5cb8-0470-46ca-8ae4-45af3fdd6247 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:19:19.652662 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:19:19.697755 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa92d4-2ba5-4f3d-a5f0-a49b03b89054 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:19:19.698508 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa92d4-2ba5-4f3d-a5f0-a49b03b89054 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:19.698879 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa92d4-2ba5-4f3d-a5f0-a49b03b89054 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:19.699274 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa92d4-2ba5-4f3d-a5f0-a49b03b89054 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:19.700993 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-22fa92d4-2ba5-4f3d-a5f0-a49b03b89054 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:19.796507 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fa92d4-2ba5-4f3d-a5f0-a49b03b89054 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:19:19.798283 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fa92d4-2ba5-4f3d-a5f0-a49b03b89054 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:19:19.817199 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 684/1360] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:19:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:19:22.025868 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb7ec29-f12c-4ea6-9c9a-4fb37abb225a None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:22.026397 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb7ec29-f12c-4ea6-9c9a-4fb37abb225a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:22.026397 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb7ec29-f12c-4ea6-9c9a-4fb37abb225a None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:22.027983 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 677/1361] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:19:22.033531 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c79d8a-8da3-4ce1-809f-499232839f86 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:22.034001 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c79d8a-8da3-4ce1-809f-499232839f86 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:22.034093 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c79d8a-8da3-4ce1-809f-499232839f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:22.104418 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-03c79d8a-8da3-4ce1-809f-499232839f86 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:19:22.105617 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-03c79d8a-8da3-4ce1-809f-499232839f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:22.145317 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:19:22.211355 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 685/1362] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:19:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:19:22.257321 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb7ec29-f12c-4ea6-9c9a-4fb37abb225a None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:19:22.259053 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6bb7ec29-f12c-4ea6-9c9a-4fb37abb225a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:22.357232 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:19:22.453229 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb7ec29-f12c-4ea6-9c9a-4fb37abb225a None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:19:22.609119 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0187b912-f2d0-454a-868d-7e74afa5d5f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:19:22.609746 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0187b912-f2d0-454a-868d-7e74afa5d5f9 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:22.610024 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0187b912-f2d0-454a-868d-7e74afa5d5f9 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:22.610291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0187b912-f2d0-454a-868d-7e74afa5d5f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:22.612043 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0187b912-f2d0-454a-868d-7e74afa5d5f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:22.635840 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:19:22.709315 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0187b912-f2d0-454a-868d-7e74afa5d5f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:19:22.712146 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0187b912-f2d0-454a-868d-7e74afa5d5f9 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:19:22.741521 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 678/1363] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:19:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:19:25.099147 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e6e76-7f7a-49e6-a266-04a5f94a04de None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:25.100008 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e6e76-7f7a-49e6-a266-04a5f94a04de None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:25.100281 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e6e76-7f7a-49e6-a266-04a5f94a04de None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:25.101937 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 686/1364] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:19:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:19:25.107203 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db89322f-a6eb-4fba-8365-3b934881692c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:25.107659 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db89322f-a6eb-4fba-8365-3b934881692c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:25.107867 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db89322f-a6eb-4fba-8365-3b934881692c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:25.195979 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-db89322f-a6eb-4fba-8365-3b934881692c None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:19:25.197190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-db89322f-a6eb-4fba-8365-3b934881692c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:25.303130 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:19:25.310937 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 679/1365] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:19:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:19:25.353965 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17649c-994c-413a-a0b0-2d3409a1e1a0 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:25.354679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17649c-994c-413a-a0b0-2d3409a1e1a0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:25.355057 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17649c-994c-413a-a0b0-2d3409a1e1a0 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:25.356819 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 687/1366] 10.222.0.40 () {54 vars in 962 bytes} [Fri Aug 5 10:19:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:19:25.362689 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d45fb-f4b7-409f-983c-ddc0c2f95fcb None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:25.363024 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d45fb-f4b7-409f-983c-ddc0c2f95fcb None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:25.363432 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d45fb-f4b7-409f-983c-ddc0c2f95fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:25.456822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-340d45fb-f4b7-409f-983c-ddc0c2f95fcb None None] MFA Rules not processed for user `b938f1f41db94b0eafe95bc8eeaae175`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:19:25.458045 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-340d45fb-f4b7-409f-983c-ddc0c2f95fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:25.583133 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 680/1367] 10.222.0.40 () {58 vars in 1059 bytes} [Fri Aug 5 10:19:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:19:25.590924 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17649c-994c-413a-a0b0-2d3409a1e1a0 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:19:25.592542 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2a17649c-994c-413a-a0b0-2d3409a1e1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:25.779208 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17649c-994c-413a-a0b0-2d3409a1e1a0 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:19:25.802474 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:19:25.905623 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e6659-87df-455d-94f5-8f38d86753b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:19:25.906263 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e6659-87df-455d-94f5-8f38d86753b2 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:25.906542 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e6659-87df-455d-94f5-8f38d86753b2 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:25.906810 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e6659-87df-455d-94f5-8f38d86753b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:25.908467 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c79e6659-87df-455d-94f5-8f38d86753b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:26.006862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79e6659-87df-455d-94f5-8f38d86753b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:19:26.008452 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79e6659-87df-455d-94f5-8f38d86753b2 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:19:26.031726 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 688/1368] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:19:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:19:26.889857 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9aa92-c24e-4b0c-ba4e-1af40f664b2a None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:26.890466 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9aa92-c24e-4b0c-ba4e-1af40f664b2a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:26.890466 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9aa92-c24e-4b0c-ba4e-1af40f664b2a None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:26.892065 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 681/1369] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:19:26.897745 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f10fa-270d-4d58-a73b-8106bb6669a5 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:26.898220 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f10fa-270d-4d58-a73b-8106bb6669a5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:26.898303 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f10fa-270d-4d58-a73b-8106bb6669a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:26.934021 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:19:26.973558 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-aa7f10fa-270d-4d58-a73b-8106bb6669a5 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:19:26.974795 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-aa7f10fa-270d-4d58-a73b-8106bb6669a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:27.075214 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 689/1370] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:19:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:19:27.420019 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70828449-eec7-4e34-97c2-0dbae6438c77 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:27.420371 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70828449-eec7-4e34-97c2-0dbae6438c77 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:27.420654 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70828449-eec7-4e34-97c2-0dbae6438c77 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:27.422267 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 682/1371] 10.222.0.40 () {54 vars in 964 bytes} [Fri Aug 5 10:19:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:19:27.427021 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23823d4-2abd-40e7-b842-8e08f4efd6ba None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:27.427315 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23823d4-2abd-40e7-b842-8e08f4efd6ba None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:27.427592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23823d4-2abd-40e7-b842-8e08f4efd6ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:27.510192 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c23823d4-2abd-40e7-b842-8e08f4efd6ba None None] MFA Rules not processed for user `f70369fd5b334d888de0c091d8885eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:19:27.511444 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c23823d4-2abd-40e7-b842-8e08f4efd6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:27.561744 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:19:27.594310 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 690/1372] 10.222.0.40 () {58 vars in 1061 bytes} [Fri Aug 5 10:19:27 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:19:27.601881 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70828449-eec7-4e34-97c2-0dbae6438c77 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:19:27.603429 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-70828449-eec7-4e34-97c2-0dbae6438c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:27.657429 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:19:27.718774 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70828449-eec7-4e34-97c2-0dbae6438c77 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:19:27.797655 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb8a01f-6250-48cd-965b-ec9446844737 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:19:27.798286 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb8a01f-6250-48cd-965b-ec9446844737 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:27.798568 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb8a01f-6250-48cd-965b-ec9446844737 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:27.798842 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb8a01f-6250-48cd-965b-ec9446844737 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:27.800525 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4fb8a01f-6250-48cd-965b-ec9446844737 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:27.905715 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:19:27.920680 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb8a01f-6250-48cd-965b-ec9446844737 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:19:27.922147 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb8a01f-6250-48cd-965b-ec9446844737 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:19:27.941901 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 683/1373] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:19:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:19:28.183692 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a9356-ecfe-48e8-8b53-ff4dc8327156 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:28.183692 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a9356-ecfe-48e8-8b53-ff4dc8327156 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:28.184344 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a9356-ecfe-48e8-8b53-ff4dc8327156 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:28.189905 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 691/1374] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:19:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:19:28.191143 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a22b9be-f2f0-4ca8-b26f-255319c28bb7 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:28.191474 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a22b9be-f2f0-4ca8-b26f-255319c28bb7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:28.191756 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a22b9be-f2f0-4ca8-b26f-255319c28bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:28.261482 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-0a22b9be-f2f0-4ca8-b26f-255319c28bb7 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:19:28.262553 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0a22b9be-f2f0-4ca8-b26f-255319c28bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:28.367727 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 684/1375] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:19:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:19:28.413799 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951a9356-ecfe-48e8-8b53-ff4dc8327156 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:19:28.415320 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-951a9356-ecfe-48e8-8b53-ff4dc8327156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:28.571530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951a9356-ecfe-48e8-8b53-ff4dc8327156 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:19:28.627268 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:19:28.689970 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffce620-af72-4630-94a1-a47d4c19502f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:19:28.690519 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffce620-af72-4630-94a1-a47d4c19502f service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:28.690927 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffce620-af72-4630-94a1-a47d4c19502f service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:28.691218 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffce620-af72-4630-94a1-a47d4c19502f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:28.692866 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8ffce620-af72-4630-94a1-a47d4c19502f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:28.784566 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffce620-af72-4630-94a1-a47d4c19502f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:19:28.786196 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffce620-af72-4630-94a1-a47d4c19502f service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:19:28.804612 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 692/1376] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:19:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:19:30.824515 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17574d48-0924-409b-8602-ad5fb9b5e201 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:30.825043 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17574d48-0924-409b-8602-ad5fb9b5e201 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:30.825043 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17574d48-0924-409b-8602-ad5fb9b5e201 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:30.827114 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 685/1377] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:19:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:19:30.832388 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9a837-1962-4114-ade3-bfe3fe3999da None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:30.832833 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9a837-1962-4114-ade3-bfe3fe3999da None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:30.832928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9a837-1962-4114-ade3-bfe3fe3999da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:30.918937 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-56f9a837-1962-4114-ade3-bfe3fe3999da None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:19:30.920188 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-56f9a837-1962-4114-ade3-bfe3fe3999da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:30.926927 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:19:31.020070 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 693/1378] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:19:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:19:31.064726 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17574d48-0924-409b-8602-ad5fb9b5e201 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:19:31.066377 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-17574d48-0924-409b-8602-ad5fb9b5e201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:31.131239 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:19:31.223245 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17574d48-0924-409b-8602-ad5fb9b5e201 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:19:31.368358 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b0755c-79cb-4892-86ce-59b971846769 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:19:31.368882 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0755c-79cb-4892-86ce-59b971846769 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:31.369162 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0755c-79cb-4892-86ce-59b971846769 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:31.369429 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0755c-79cb-4892-86ce-59b971846769 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:31.371081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-40b0755c-79cb-4892-86ce-59b971846769 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:31.394576 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:19:31.478498 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b0755c-79cb-4892-86ce-59b971846769 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:19:31.480160 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b0755c-79cb-4892-86ce-59b971846769 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:19:31.499101 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 686/1379] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:19:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:19:33.529282 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f070087-3f94-4d49-acbe-22761a5393df None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:33.529860 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f070087-3f94-4d49-acbe-22761a5393df None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:33.529860 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f070087-3f94-4d49-acbe-22761a5393df None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:33.531818 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 694/1380] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:19:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:19:33.536862 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f1f1ad-e003-4b61-bb54-f631873aa8a1 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:33.537303 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f1f1ad-e003-4b61-bb54-f631873aa8a1 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:33.537410 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f1f1ad-e003-4b61-bb54-f631873aa8a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:33.606811 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-63f1f1ad-e003-4b61-bb54-f631873aa8a1 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:19:33.608208 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-63f1f1ad-e003-4b61-bb54-f631873aa8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:33.708068 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:19:33.719065 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 687/1381] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:19:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:19:33.759635 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f070087-3f94-4d49-acbe-22761a5393df None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:19:33.761142 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9f070087-3f94-4d49-acbe-22761a5393df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:33.904594 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:19:33.916022 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f070087-3f94-4d49-acbe-22761a5393df None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:19:34.037448 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a3db23-e26e-4430-9a8b-891df052b92a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:19:34.038069 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3db23-e26e-4430-9a8b-891df052b92a service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:34.038342 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3db23-e26e-4430-9a8b-891df052b92a service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:34.038608 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3db23-e26e-4430-9a8b-891df052b92a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:34.040231 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-07a3db23-e26e-4430-9a8b-891df052b92a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:34.145414 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a3db23-e26e-4430-9a8b-891df052b92a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:19:34.147245 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a3db23-e26e-4430-9a8b-891df052b92a service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:19:34.171359 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:19:34.178051 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 695/1382] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:19:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 10:19:36.226953 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd0d32-fac3-495f-840b-9dfe81d8cffa None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:36.227654 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd0d32-fac3-495f-840b-9dfe81d8cffa None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:36.227654 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd0d32-fac3-495f-840b-9dfe81d8cffa None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:36.228989 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 688/1383] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:19:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:19:36.234854 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12c41c-754a-4a6a-9905-f36946feb11a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:36.235316 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12c41c-754a-4a6a-9905-f36946feb11a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:36.235438 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12c41c-754a-4a6a-9905-f36946feb11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:36.306507 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-7a12c41c-754a-4a6a-9905-f36946feb11a None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:19:36.307833 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7a12c41c-754a-4a6a-9905-f36946feb11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:36.408792 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 696/1384] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:19:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:19:36.452478 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fd0d32-fac3-495f-840b-9dfe81d8cffa None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:19:36.453975 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c1fd0d32-fac3-495f-840b-9dfe81d8cffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:36.647913 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fd0d32-fac3-495f-840b-9dfe81d8cffa None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:19:36.756524 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:19:36.840728 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b78e41-5bc3-4279-b7ef-5122c0e3004d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:19:36.842061 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b78e41-5bc3-4279-b7ef-5122c0e3004d service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:36.842437 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b78e41-5bc3-4279-b7ef-5122c0e3004d service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:36.843037 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b78e41-5bc3-4279-b7ef-5122c0e3004d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:36.846205 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d2b78e41-5bc3-4279-b7ef-5122c0e3004d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:36.951084 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b78e41-5bc3-4279-b7ef-5122c0e3004d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:19:36.952555 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b78e41-5bc3-4279-b7ef-5122c0e3004d service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:19:36.971572 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 689/1385] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:19:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 10:19:38.956815 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417959b-fee3-40a8-9b60-bea943389bcb None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:38.957399 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417959b-fee3-40a8-9b60-bea943389bcb None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:38.957399 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417959b-fee3-40a8-9b60-bea943389bcb None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:38.958729 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 697/1386] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:19:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:19:38.964173 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf2e18a-d832-49e0-9a75-cb1885f744ee None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:38.964591 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf2e18a-d832-49e0-9a75-cb1885f744ee None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:38.964721 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf2e18a-d832-49e0-9a75-cb1885f744ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:39.036459 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-cdf2e18a-d832-49e0-9a75-cb1885f744ee None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:19:39.037732 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-cdf2e18a-d832-49e0-9a75-cb1885f744ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:39.076658 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:19:39.185411 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 690/1387] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:19:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:19:39.231467 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7417959b-fee3-40a8-9b60-bea943389bcb None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:19:39.234818 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7417959b-fee3-40a8-9b60-bea943389bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:39.348321 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:19:39.433600 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7417959b-fee3-40a8-9b60-bea943389bcb None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:19:39.555184 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d6efd9-b3b5-4bdf-904a-1b15c4c9df95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:19:39.555969 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6efd9-b3b5-4bdf-904a-1b15c4c9df95 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:39.556328 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6efd9-b3b5-4bdf-904a-1b15c4c9df95 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:39.556699 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6efd9-b3b5-4bdf-904a-1b15c4c9df95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:39.558985 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b8d6efd9-b3b5-4bdf-904a-1b15c4c9df95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:39.611890 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:19:39.667168 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d6efd9-b3b5-4bdf-904a-1b15c4c9df95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:19:39.668829 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d6efd9-b3b5-4bdf-904a-1b15c4c9df95 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:19:39.688301 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 698/1388] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:19:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:19:42.191076 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524c70e-6e2e-47c5-b596-15ad112502b2 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:42.191614 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524c70e-6e2e-47c5-b596-15ad112502b2 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:42.191614 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524c70e-6e2e-47c5-b596-15ad112502b2 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:42.192996 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 691/1389] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:19:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:19:42.198363 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a0964-cc2f-4bb2-b3aa-4cb72013db99 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:42.198791 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a0964-cc2f-4bb2-b3aa-4cb72013db99 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:42.198997 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a0964-cc2f-4bb2-b3aa-4cb72013db99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:42.264680 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c87a0964-cc2f-4bb2-b3aa-4cb72013db99 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:19:42.265863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c87a0964-cc2f-4bb2-b3aa-4cb72013db99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:42.363546 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 699/1390] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:19:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:19:42.409068 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1524c70e-6e2e-47c5-b596-15ad112502b2 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:19:42.410675 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1524c70e-6e2e-47c5-b596-15ad112502b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:42.562551 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1524c70e-6e2e-47c5-b596-15ad112502b2 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:19:42.579425 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:19:42.681965 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a60f49e-7751-48a9-a5f7-580d27735350 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:19:42.682497 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a60f49e-7751-48a9-a5f7-580d27735350 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:42.682770 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a60f49e-7751-48a9-a5f7-580d27735350 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:42.683069 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a60f49e-7751-48a9-a5f7-580d27735350 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:42.684689 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2a60f49e-7751-48a9-a5f7-580d27735350 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:42.775305 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a60f49e-7751-48a9-a5f7-580d27735350 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:19:42.776951 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a60f49e-7751-48a9-a5f7-580d27735350 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:19:42.795534 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 692/1391] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:19:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:19:42.956077 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e99d4-1886-4542-8243-7964b5bd3f8a None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:42.956423 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e99d4-1886-4542-8243-7964b5bd3f8a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:42.956721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e99d4-1886-4542-8243-7964b5bd3f8a None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:42.958142 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 700/1392] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:19:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:19:42.964290 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2aa72c-7fc7-4bb3-9fc4-2f059dcf4a57 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:42.964604 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2aa72c-7fc7-4bb3-9fc4-2f059dcf4a57 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:42.964877 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2aa72c-7fc7-4bb3-9fc4-2f059dcf4a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:42.993549 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:19:43.038619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ed2aa72c-7fc7-4bb3-9fc4-2f059dcf4a57 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:19:43.039863 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ed2aa72c-7fc7-4bb3-9fc4-2f059dcf4a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:43.142095 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 693/1393] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:19:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:19:43.199447 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ddaeb-4dd8-42b0-94b6-36ee018bf124 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:43.199986 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ddaeb-4dd8-42b0-94b6-36ee018bf124 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:43.200071 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ddaeb-4dd8-42b0-94b6-36ee018bf124 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:43.201717 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 701/1394] 10.222.0.40 () {54 vars in 966 bytes} [Fri Aug 5 10:19:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:19:43.206806 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ee9ee1-fe97-468f-aee5-0b32d524e983 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:43.207227 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ee9ee1-fe97-468f-aee5-0b32d524e983 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:43.207538 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ee9ee1-fe97-468f-aee5-0b32d524e983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:43.287318 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-d1ee9ee1-fe97-468f-aee5-0b32d524e983 None None] MFA Rules not processed for user `ac0a8cf7f8b74e88a04444ec38547528`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:19:43.288536 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d1ee9ee1-fe97-468f-aee5-0b32d524e983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:43.333329 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:19:43.384142 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 694/1395] 10.222.0.40 () {58 vars in 1063 bytes} [Fri Aug 5 10:19:43 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:19:43.392364 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ddaeb-4dd8-42b0-94b6-36ee018bf124 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:19:43.393990 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-226ddaeb-4dd8-42b0-94b6-36ee018bf124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:43.409222 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:19:43.502610 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ddaeb-4dd8-42b0-94b6-36ee018bf124 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:19:43.580855 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c966553-33f9-4869-8090-b67bf110b0cf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac0a8cf7f8b74e88a04444ec38547528', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:19:43.581476 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c966553-33f9-4869-8090-b67bf110b0cf service glance] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:43.581749 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c966553-33f9-4869-8090-b67bf110b0cf service glance] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:43.582018 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c966553-33f9-4869-8090-b67bf110b0cf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:43.583670 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6c966553-33f9-4869-8090-b67bf110b0cf service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:43.654691 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:19:43.697797 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c966553-33f9-4869-8090-b67bf110b0cf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:19:43.699566 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c966553-33f9-4869-8090-b67bf110b0cf service glance] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:19:43.719969 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 702/1396] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:19:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:19:44.469535 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b272c754-d668-4d7f-bc24-09627fdf5bd9 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:44.470084 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b272c754-d668-4d7f-bc24-09627fdf5bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:44.470192 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b272c754-d668-4d7f-bc24-09627fdf5bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:44.538484 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b272c754-d668-4d7f-bc24-09627fdf5bd9 None None] MFA Rules not processed for user `52fc3ba90dc045c2a5b7bb730da96acc`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:19:44.539744 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b272c754-d668-4d7f-bc24-09627fdf5bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:44.626510 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 695/1397] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 5 10:19:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:19:44.669298 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde0813-6975-4168-96b6-be2b0dd712cd None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:44.669849 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde0813-6975-4168-96b6-be2b0dd712cd None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:44.669849 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde0813-6975-4168-96b6-be2b0dd712cd None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:44.671286 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 703/1398] 10.222.0.40 () {54 vars in 943 bytes} [Fri Aug 5 10:19:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:19:44.677640 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e809a0-6599-464f-ba6f-078c69693c63 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:44.677923 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e809a0-6599-464f-ba6f-078c69693c63 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:44.678194 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e809a0-6599-464f-ba6f-078c69693c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:44.745774 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:19:44.755248 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-33e809a0-6599-464f-ba6f-078c69693c63 None None] MFA Rules not processed for user `c80cf0e429f04f229aea1aa9293a93b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:19:44.756447 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-33e809a0-6599-464f-ba6f-078c69693c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:44.843179 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f6dc4d-aed5-421a-a6ab-07c8e652ff83 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:44.843441 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f6dc4d-aed5-421a-a6ab-07c8e652ff83 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:44.843727 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f6dc4d-aed5-421a-a6ab-07c8e652ff83 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:44.845093 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 704/1399] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:19:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 10:19:44.850120 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8feaa1-622c-403f-9d79-d0dc757ea965 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:44.850403 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8feaa1-622c-403f-9d79-d0dc757ea965 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:44.850679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8feaa1-622c-403f-9d79-d0dc757ea965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:44.863328 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 696/1400] 10.222.0.40 () {58 vars in 1040 bytes} [Fri Aug 5 10:19:44 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:19:44.873191 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e809a0-6599-464f-ba6f-078c69693c63 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:19:44.883573 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e809a0-6599-464f-ba6f-078c69693c63 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:19:44.923363 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-3b8feaa1-622c-403f-9d79-d0dc757ea965 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:19:44.924543 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3b8feaa1-622c-403f-9d79-d0dc757ea965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:44.994631 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d5c81-2848-4008-9c61-f5e2811b43cd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c80cf0e429f04f229aea1aa9293a93b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:19:44.995390 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d5c81-2848-4008-9c61-f5e2811b43cd service swift] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:44.995667 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d5c81-2848-4008-9c61-f5e2811b43cd service swift] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:44.995985 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d5c81-2848-4008-9c61-f5e2811b43cd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:44.997910 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9c6d5c81-2848-4008-9c61-f5e2811b43cd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:45.000216 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:19:45.029671 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 705/1401] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:19:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:19:45.074134 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8feaa1-622c-403f-9d79-d0dc757ea965 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:19:45.075915 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3b8feaa1-622c-403f-9d79-d0dc757ea965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:45.084472 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d5c81-2848-4008-9c61-f5e2811b43cd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:19:45.086069 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d5c81-2848-4008-9c61-f5e2811b43cd service swift] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:19:45.094345 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 697/1402] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 5 10:19:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 05 10:19:45.102379 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:19:45.200706 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8feaa1-622c-403f-9d79-d0dc757ea965 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:19:45.340214 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0474b584-e4c4-4a84-aa04-332320b48227 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:19:45.340716 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0474b584-e4c4-4a84-aa04-332320b48227 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:45.341561 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0474b584-e4c4-4a84-aa04-332320b48227 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:45.341847 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0474b584-e4c4-4a84-aa04-332320b48227 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:45.343702 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0474b584-e4c4-4a84-aa04-332320b48227 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:45.363924 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:19:45.444511 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0474b584-e4c4-4a84-aa04-332320b48227 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:19:45.446115 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0474b584-e4c4-4a84-aa04-332320b48227 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:19:45.469323 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 706/1403] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:19:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 10:19:47.511041 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57352f5-4091-4b46-aeeb-061f4957bb0f None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:47.511616 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57352f5-4091-4b46-aeeb-061f4957bb0f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:47.511616 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57352f5-4091-4b46-aeeb-061f4957bb0f None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:47.514942 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 698/1404] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:19:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:19:47.518470 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eaf0fd-d960-4f59-a28d-50b185b4bec8 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:47.518934 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eaf0fd-d960-4f59-a28d-50b185b4bec8 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:47.519035 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eaf0fd-d960-4f59-a28d-50b185b4bec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:47.591707 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-83eaf0fd-d960-4f59-a28d-50b185b4bec8 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:19:47.592975 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-83eaf0fd-d960-4f59-a28d-50b185b4bec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:47.700754 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:19:47.707722 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 707/1405] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:19:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:19:47.753661 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57352f5-4091-4b46-aeeb-061f4957bb0f None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:19:47.755241 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a57352f5-4091-4b46-aeeb-061f4957bb0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:47.934538 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:19:47.938454 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57352f5-4091-4b46-aeeb-061f4957bb0f None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:19:48.080971 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc622fc-cc66-48da-a4cd-a06d39c817b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:19:48.081671 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc622fc-cc66-48da-a4cd-a06d39c817b1 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:48.081950 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc622fc-cc66-48da-a4cd-a06d39c817b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:48.082222 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc622fc-cc66-48da-a4cd-a06d39c817b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:48.083886 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-adc622fc-cc66-48da-a4cd-a06d39c817b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:48.195093 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc622fc-cc66-48da-a4cd-a06d39c817b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:19:48.196748 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc622fc-cc66-48da-a4cd-a06d39c817b1 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:19:48.219651 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 699/1406] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:19:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:19:50.253407 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a9ab97-87f5-42e4-b7fb-2f4910ead5c4 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:50.253999 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a9ab97-87f5-42e4-b7fb-2f4910ead5c4 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:50.253999 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a9ab97-87f5-42e4-b7fb-2f4910ead5c4 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:50.255524 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 708/1407] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:19:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:19:50.262925 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7991290-fba0-42b5-ba80-c39fb71974ec None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:50.262925 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7991290-fba0-42b5-ba80-c39fb71974ec None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:50.262925 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7991290-fba0-42b5-ba80-c39fb71974ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:50.267402 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:19:50.334929 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-f7991290-fba0-42b5-ba80-c39fb71974ec None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:19:50.336202 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f7991290-fba0-42b5-ba80-c39fb71974ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:50.454873 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 700/1408] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:19:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:19:50.503640 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a9ab97-87f5-42e4-b7fb-2f4910ead5c4 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:19:50.505472 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-27a9ab97-87f5-42e4-b7fb-2f4910ead5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:50.669511 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a9ab97-87f5-42e4-b7fb-2f4910ead5c4 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:19:50.741871 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:19:50.795211 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150a55d-ae6c-44fa-ba72-76549d6153a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:19:50.795931 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150a55d-ae6c-44fa-ba72-76549d6153a1 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:19:50.796286 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150a55d-ae6c-44fa-ba72-76549d6153a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:19:50.796637 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150a55d-ae6c-44fa-ba72-76549d6153a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:19:50.798985 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9150a55d-ae6c-44fa-ba72-76549d6153a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:19:50.906184 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9150a55d-ae6c-44fa-ba72-76549d6153a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:19:50.907850 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9150a55d-ae6c-44fa-ba72-76549d6153a1 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:19:50.927679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 709/1409] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:19:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:20:00.452679 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=64 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:20:00.924736 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=64 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:20:07.415315 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c7ff5-6e41-4477-bb5a-9ca7b7bc15e8 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:07.415852 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c7ff5-6e41-4477-bb5a-9ca7b7bc15e8 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:07.415852 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c7ff5-6e41-4477-bb5a-9ca7b7bc15e8 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:07.417392 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 701/1410] 10.222.0.40 () {54 vars in 918 bytes} [Fri Aug 5 10:20:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:20:07.423671 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f4fdc3-f88a-43f5-9314-4f389fe82ed2 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:07.424140 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f4fdc3-f88a-43f5-9314-4f389fe82ed2 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:07.424394 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f4fdc3-f88a-43f5-9314-4f389fe82ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:07.508871 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-29f4fdc3-f88a-43f5-9314-4f389fe82ed2 None None] MFA Rules not processed for user `8743f5bb0be54d57a957617636b0b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:20:07.510105 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-29f4fdc3-f88a-43f5-9314-4f389fe82ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:07.620162 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 710/1411] 10.222.0.40 () {58 vars in 1015 bytes} [Fri Aug 5 10:20:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:20:07.636493 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce4b6c-1fcc-4e42-94be-462f67b27a01 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:07.636861 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce4b6c-1fcc-4e42-94be-462f67b27a01 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:07.637272 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce4b6c-1fcc-4e42-94be-462f67b27a01 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:07.639542 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 702/1412] 10.222.0.40 () {54 vars in 918 bytes} [Fri Aug 5 10:20:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:20:07.644483 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba04855d-9b26-4b5c-914c-e13e8a2cfa31 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:07.644778 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba04855d-9b26-4b5c-914c-e13e8a2cfa31 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:07.645118 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba04855d-9b26-4b5c-914c-e13e8a2cfa31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:07.721785 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:20:07.725570 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ba04855d-9b26-4b5c-914c-e13e8a2cfa31 None None] MFA Rules not processed for user `b938f1f41db94b0eafe95bc8eeaae175`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:20:07.727449 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ba04855d-9b26-4b5c-914c-e13e8a2cfa31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:07.850684 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 711/1413] 10.222.0.40 () {58 vars in 1015 bytes} [Fri Aug 5 10:20:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:20:10.325635 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e7e62-94ca-4ace-9a81-3e0442b7548d None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:10.326255 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e7e62-94ca-4ace-9a81-3e0442b7548d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:10.326255 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e7e62-94ca-4ace-9a81-3e0442b7548d None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:10.327578 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 703/1414] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 10:20:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:20:10.333763 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19650cfa-15a3-46d7-8188-0772d8905c23 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:10.334504 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19650cfa-15a3-46d7-8188-0772d8905c23 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:10.334747 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19650cfa-15a3-46d7-8188-0772d8905c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:10.414045 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-19650cfa-15a3-46d7-8188-0772d8905c23 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:20:10.415312 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-19650cfa-15a3-46d7-8188-0772d8905c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:10.486376 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:20:10.531591 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 712/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 10:20:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:20:10.580265 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e7e62-94ca-4ace-9a81-3e0442b7548d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:20:10.582372 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e80e7e62-94ca-4ace-9a81-3e0442b7548d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:10.738200 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e7e62-94ca-4ace-9a81-3e0442b7548d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:20:10.821817 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:20:10.863000 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a6dc7f-f516-4c0e-adc4-dfe267717013 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:20:10.863571 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6dc7f-f516-4c0e-adc4-dfe267717013 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:10.863837 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6dc7f-f516-4c0e-adc4-dfe267717013 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:10.864099 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6dc7f-f516-4c0e-adc4-dfe267717013 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:10.865718 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d4a6dc7f-f516-4c0e-adc4-dfe267717013 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:10.970252 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a6dc7f-f516-4c0e-adc4-dfe267717013 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:20:10.971907 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a6dc7f-f516-4c0e-adc4-dfe267717013 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:20:10.991233 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 704/1416] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:20:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:20:11.919293 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa7ee8-a085-4bb2-9113-75f310eb2fee None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:11.919904 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa7ee8-a085-4bb2-9113-75f310eb2fee None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:11.919904 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa7ee8-a085-4bb2-9113-75f310eb2fee None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:11.921229 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 713/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 10:20:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:20:11.927255 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b28639-64f3-4990-a7db-a7940f73a5f5 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:11.927932 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b28639-64f3-4990-a7db-a7940f73a5f5 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:11.928091 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b28639-64f3-4990-a7db-a7940f73a5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:12.147631 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-84b28639-64f3-4990-a7db-a7940f73a5f5 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:20:12.148839 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-84b28639-64f3-4990-a7db-a7940f73a5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:12.190443 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:20:12.263912 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 705/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 10:20:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:20:12.417862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aa7ee8-a085-4bb2-9113-75f310eb2fee None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:20:12.420212 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-60aa7ee8-a085-4bb2-9113-75f310eb2fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:12.595548 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aa7ee8-a085-4bb2-9113-75f310eb2fee None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:20:12.641770 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:20:12.716904 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab1d047-4f3b-4d4e-8d51-0fea75cfdd54 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:20:12.717522 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab1d047-4f3b-4d4e-8d51-0fea75cfdd54 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:12.717799 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab1d047-4f3b-4d4e-8d51-0fea75cfdd54 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:12.718073 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab1d047-4f3b-4d4e-8d51-0fea75cfdd54 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:12.719730 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6ab1d047-4f3b-4d4e-8d51-0fea75cfdd54 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:12.817755 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab1d047-4f3b-4d4e-8d51-0fea75cfdd54 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:20:12.819395 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab1d047-4f3b-4d4e-8d51-0fea75cfdd54 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:20:12.839410 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 714/1419] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:20:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:20:13.720941 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae11bc60-9675-4721-bad1-ae291de5fc36 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:13.721551 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae11bc60-9675-4721-bad1-ae291de5fc36 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:13.721551 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae11bc60-9675-4721-bad1-ae291de5fc36 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:13.722828 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 706/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 10:20:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:20:13.730857 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e57aaec-daa6-46e3-b94c-faac0e4b47d6 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:13.731329 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e57aaec-daa6-46e3-b94c-faac0e4b47d6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:13.731471 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e57aaec-daa6-46e3-b94c-faac0e4b47d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:13.793214 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:20:13.820087 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-3e57aaec-daa6-46e3-b94c-faac0e4b47d6 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:20:13.821325 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3e57aaec-daa6-46e3-b94c-faac0e4b47d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:13.928717 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 715/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 10:20:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:20:14.098463 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae11bc60-9675-4721-bad1-ae291de5fc36 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:20:14.100857 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ae11bc60-9675-4721-bad1-ae291de5fc36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:14.283582 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae11bc60-9675-4721-bad1-ae291de5fc36 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:20:14.296698 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:20:14.403023 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e782955-db16-40e6-aaef-c1c2294325bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:20:14.403641 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e782955-db16-40e6-aaef-c1c2294325bd service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:14.403916 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e782955-db16-40e6-aaef-c1c2294325bd service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:14.404185 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e782955-db16-40e6-aaef-c1c2294325bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:14.405793 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6e782955-db16-40e6-aaef-c1c2294325bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:14.505134 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e782955-db16-40e6-aaef-c1c2294325bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:20:14.506771 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e782955-db16-40e6-aaef-c1c2294325bd service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:20:14.525197 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 707/1422] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:20:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:20:15.429291 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09511fad-e9da-4f19-a1e7-2f723f95c34b None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:15.429832 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09511fad-e9da-4f19-a1e7-2f723f95c34b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:15.429832 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09511fad-e9da-4f19-a1e7-2f723f95c34b None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:15.431182 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 716/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 10:20:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 10:20:15.436928 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f000bf29-d14d-4a6c-9190-ef5385d51b95 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:15.437387 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f000bf29-d14d-4a6c-9190-ef5385d51b95 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:15.437480 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f000bf29-d14d-4a6c-9190-ef5385d51b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:15.462220 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:20:15.529207 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-f000bf29-d14d-4a6c-9190-ef5385d51b95 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:20:15.530423 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f000bf29-d14d-4a6c-9190-ef5385d51b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:15.637898 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 708/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 10:20:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:20:15.789760 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09511fad-e9da-4f19-a1e7-2f723f95c34b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:20:15.791345 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-09511fad-e9da-4f19-a1e7-2f723f95c34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:15.933970 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:20:15.982307 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09511fad-e9da-4f19-a1e7-2f723f95c34b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:20:16.130397 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8136d1d4-8e6d-4b4c-a9b4-799361e16245 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:20:16.130799 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136d1d4-8e6d-4b4c-a9b4-799361e16245 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:16.131110 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136d1d4-8e6d-4b4c-a9b4-799361e16245 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:16.131940 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136d1d4-8e6d-4b4c-a9b4-799361e16245 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:16.135105 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8136d1d4-8e6d-4b4c-a9b4-799361e16245 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:16.224524 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:20:16.240864 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8136d1d4-8e6d-4b4c-a9b4-799361e16245 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:20:16.242531 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8136d1d4-8e6d-4b4c-a9b4-799361e16245 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:20:16.263568 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 717/1425] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:20:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:20:17.168182 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98148cf2-21cc-48d9-ad3c-ab9e31348798 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:17.168627 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98148cf2-21cc-48d9-ad3c-ab9e31348798 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:17.168715 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98148cf2-21cc-48d9-ad3c-ab9e31348798 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:17.170125 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 709/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 10:20:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:20:17.175331 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d4f95-94f0-40c3-a3a9-fd8f81c51896 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:17.175765 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d4f95-94f0-40c3-a3a9-fd8f81c51896 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:17.175867 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d4f95-94f0-40c3-a3a9-fd8f81c51896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:17.252242 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a77d4f95-94f0-40c3-a3a9-fd8f81c51896 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:20:17.253461 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a77d4f95-94f0-40c3-a3a9-fd8f81c51896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:17.358217 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 718/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 10:20:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:20:17.523796 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98148cf2-21cc-48d9-ad3c-ab9e31348798 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:20:17.526225 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-98148cf2-21cc-48d9-ad3c-ab9e31348798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:17.624842 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:20:17.696384 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98148cf2-21cc-48d9-ad3c-ab9e31348798 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:20:17.814108 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f3e83d-045c-4aba-8d2f-5a6a9a8cd97e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:20:17.814642 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f3e83d-045c-4aba-8d2f-5a6a9a8cd97e service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:17.814939 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f3e83d-045c-4aba-8d2f-5a6a9a8cd97e service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:17.815212 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f3e83d-045c-4aba-8d2f-5a6a9a8cd97e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:17.816842 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c0f3e83d-045c-4aba-8d2f-5a6a9a8cd97e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:17.868985 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:20:17.920599 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f3e83d-045c-4aba-8d2f-5a6a9a8cd97e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:20:17.922239 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f3e83d-045c-4aba-8d2f-5a6a9a8cd97e service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:20:17.941613 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 710/1428] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:20:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:20:18.787306 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d55f61-c02f-4504-b600-7b8478192682 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:18.787992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d55f61-c02f-4504-b600-7b8478192682 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:18.787992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d55f61-c02f-4504-b600-7b8478192682 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:18.789221 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 719/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 10:20:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:20:18.794993 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb68fec-b6f8-405e-91e3-35f32d242285 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:18.795449 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb68fec-b6f8-405e-91e3-35f32d242285 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:18.795558 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb68fec-b6f8-405e-91e3-35f32d242285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:18.879493 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-bcb68fec-b6f8-405e-91e3-35f32d242285 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:20:18.880709 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-bcb68fec-b6f8-405e-91e3-35f32d242285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:18.988163 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 711/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 10:20:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:20:19.150875 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d55f61-c02f-4504-b600-7b8478192682 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:20:19.152424 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-44d55f61-c02f-4504-b600-7b8478192682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:19.209305 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:20:19.323752 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d55f61-c02f-4504-b600-7b8478192682 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:20:19.445188 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eeafea-19b0-40d8-9c2f-3f4fd32f978d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:20:19.445736 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eeafea-19b0-40d8-9c2f-3f4fd32f978d service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:19.446017 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eeafea-19b0-40d8-9c2f-3f4fd32f978d service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:19.446292 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eeafea-19b0-40d8-9c2f-3f4fd32f978d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:19.447979 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-24eeafea-19b0-40d8-9c2f-3f4fd32f978d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:19.455289 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:20:19.542490 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eeafea-19b0-40d8-9c2f-3f4fd32f978d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:20:19.544206 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eeafea-19b0-40d8-9c2f-3f4fd32f978d service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:20:19.563415 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 720/1431] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:20:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:20:20.400585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc0567-d4ce-4324-9aff-cc226ddd5e2c None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:20.401098 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc0567-d4ce-4324-9aff-cc226ddd5e2c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:20.401098 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc0567-d4ce-4324-9aff-cc226ddd5e2c None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:20.402730 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 712/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 10:20:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:20:20.407823 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e471d3-c4ea-4361-ba75-298af18581f4 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:20.408280 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e471d3-c4ea-4361-ba75-298af18581f4 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:20.408383 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e471d3-c4ea-4361-ba75-298af18581f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:20.492039 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c0e471d3-c4ea-4361-ba75-298af18581f4 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:20:20.493258 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c0e471d3-c4ea-4361-ba75-298af18581f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:20.573937 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:20:20.610594 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 721/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 10:20:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:20:20.763028 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc0567-d4ce-4324-9aff-cc226ddd5e2c None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:20:20.764500 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-40dc0567-d4ce-4324-9aff-cc226ddd5e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:20.791921 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:20:20.924853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc0567-d4ce-4324-9aff-cc226ddd5e2c None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:20:21.020578 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:20:21.049679 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a38c015-023f-49a5-892c-1258388ebce9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:20:21.050304 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a38c015-023f-49a5-892c-1258388ebce9 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:21.050541 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a38c015-023f-49a5-892c-1258388ebce9 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:21.050839 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a38c015-023f-49a5-892c-1258388ebce9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:21.052488 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7a38c015-023f-49a5-892c-1258388ebce9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:21.162588 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a38c015-023f-49a5-892c-1258388ebce9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:20:21.164301 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a38c015-023f-49a5-892c-1258388ebce9 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:20:21.183427 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 713/1434] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:20:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:20:21.995605 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a9e980-8c31-44d6-be5d-8933d603a037 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:21.996472 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a9e980-8c31-44d6-be5d-8933d603a037 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:21.996573 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a9e980-8c31-44d6-be5d-8933d603a037 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:21.998041 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 722/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 10:20:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:20:22.003389 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e810b-ea5a-40f6-bc33-1e1e90592527 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:22.003836 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e810b-ea5a-40f6-bc33-1e1e90592527 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:22.003940 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e810b-ea5a-40f6-bc33-1e1e90592527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:22.084742 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-dd6e810b-ea5a-40f6-bc33-1e1e90592527 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:20:22.085962 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-dd6e810b-ea5a-40f6-bc33-1e1e90592527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:22.129854 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:20:22.190781 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 714/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 10:20:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:20:22.343417 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a9e980-8c31-44d6-be5d-8933d603a037 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:20:22.344873 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f5a9e980-8c31-44d6-be5d-8933d603a037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:22.512647 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a9e980-8c31-44d6-be5d-8933d603a037 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:20:22.575366 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:20:22.644664 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd52ab6-442d-4add-8cd5-f404ca7def76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:20:22.645208 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd52ab6-442d-4add-8cd5-f404ca7def76 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:22.645490 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd52ab6-442d-4add-8cd5-f404ca7def76 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:22.645778 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd52ab6-442d-4add-8cd5-f404ca7def76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:22.647815 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ccd52ab6-442d-4add-8cd5-f404ca7def76 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:22.746496 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd52ab6-442d-4add-8cd5-f404ca7def76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:20:22.748347 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd52ab6-442d-4add-8cd5-f404ca7def76 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:20:22.766602 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 723/1437] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:20:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:20:23.572532 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f5d1d-594b-4fb3-84dd-8e9a8b30f274 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:23.573105 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f5d1d-594b-4fb3-84dd-8e9a8b30f274 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:23.573105 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f5d1d-594b-4fb3-84dd-8e9a8b30f274 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:23.574464 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 715/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 10:20:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:20:23.580563 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f45007-209b-4b59-9ec3-0c95b68d4b84 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:23.581335 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f45007-209b-4b59-9ec3-0c95b68d4b84 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:23.581472 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f45007-209b-4b59-9ec3-0c95b68d4b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:23.662087 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-74f45007-209b-4b59-9ec3-0c95b68d4b84 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:20:23.663298 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-74f45007-209b-4b59-9ec3-0c95b68d4b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:23.665421 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:20:23.776052 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 724/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 10:20:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:20:23.925468 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5f5d1d-594b-4fb3-84dd-8e9a8b30f274 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:20:23.927007 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-df5f5d1d-594b-4fb3-84dd-8e9a8b30f274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:24.089158 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5f5d1d-594b-4fb3-84dd-8e9a8b30f274 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:20:24.118013 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:20:24.212668 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e6136e-5c48-48e1-a55f-f9c1dcce0c16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:20:24.213291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e6136e-5c48-48e1-a55f-f9c1dcce0c16 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:24.213568 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e6136e-5c48-48e1-a55f-f9c1dcce0c16 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:24.213835 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e6136e-5c48-48e1-a55f-f9c1dcce0c16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:24.215546 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-03e6136e-5c48-48e1-a55f-f9c1dcce0c16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:24.323196 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e6136e-5c48-48e1-a55f-f9c1dcce0c16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:20:24.324795 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e6136e-5c48-48e1-a55f-f9c1dcce0c16 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:20:24.344738 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 716/1440] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:20:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:20:26.229994 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d4f7c-fc59-47c2-ac67-cf11227cb782 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:26.230546 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d4f7c-fc59-47c2-ac67-cf11227cb782 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:26.230955 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d4f7c-fc59-47c2-ac67-cf11227cb782 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:26.232991 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 725/1441] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 10:20:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:20:26.242445 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34d477e-7b3b-436a-8bb7-0dadb1a58c2f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:26.242878 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34d477e-7b3b-436a-8bb7-0dadb1a58c2f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:26.243066 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34d477e-7b3b-436a-8bb7-0dadb1a58c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:26.290596 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:20:26.318945 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a34d477e-7b3b-436a-8bb7-0dadb1a58c2f None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:20:26.320197 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a34d477e-7b3b-436a-8bb7-0dadb1a58c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:26.422841 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 717/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 10:20:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:20:26.465469 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d4f7c-fc59-47c2-ac67-cf11227cb782 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:20:26.467117 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-be0d4f7c-fc59-47c2-ac67-cf11227cb782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:26.702075 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d4f7c-fc59-47c2-ac67-cf11227cb782 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:20:26.742628 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:20:26.786052 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab564951-66e7-4454-920c-0532605034eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:20:26.786661 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab564951-66e7-4454-920c-0532605034eb service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:26.786961 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab564951-66e7-4454-920c-0532605034eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:26.787231 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab564951-66e7-4454-920c-0532605034eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:26.788842 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ab564951-66e7-4454-920c-0532605034eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:26.901429 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab564951-66e7-4454-920c-0532605034eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:20:26.902995 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab564951-66e7-4454-920c-0532605034eb service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:20:26.921225 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 726/1443] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:20:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:20:29.071728 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac46aaf-3803-4589-97bf-c7e546f93c77 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:29.072325 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac46aaf-3803-4589-97bf-c7e546f93c77 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:29.072325 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac46aaf-3803-4589-97bf-c7e546f93c77 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:29.074423 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 718/1444] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 10:20:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:20:29.081169 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b541a-5e80-4549-9310-cbb9c17fcc59 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:29.081584 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b541a-5e80-4549-9310-cbb9c17fcc59 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:29.081726 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b541a-5e80-4549-9310-cbb9c17fcc59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:29.158934 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-4a8b541a-5e80-4549-9310-cbb9c17fcc59 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:20:29.160403 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4a8b541a-5e80-4549-9310-cbb9c17fcc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:29.174084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:20:29.265375 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 727/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 10:20:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:20:29.309127 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac46aaf-3803-4589-97bf-c7e546f93c77 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:20:29.311050 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7ac46aaf-3803-4589-97bf-c7e546f93c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:29.428619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac46aaf-3803-4589-97bf-c7e546f93c77 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:20:29.442841 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:20:29.512416 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12a0a1-4d04-4fb2-bf9c-dbdd0a6b191c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:20:29.513175 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12a0a1-4d04-4fb2-bf9c-dbdd0a6b191c service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:29.513427 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12a0a1-4d04-4fb2-bf9c-dbdd0a6b191c service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:29.513695 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12a0a1-4d04-4fb2-bf9c-dbdd0a6b191c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:29.515342 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-cc12a0a1-4d04-4fb2-bf9c-dbdd0a6b191c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:29.630107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc12a0a1-4d04-4fb2-bf9c-dbdd0a6b191c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:20:29.632397 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc12a0a1-4d04-4fb2-bf9c-dbdd0a6b191c service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:20:29.659862 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 719/1446] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:20:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:20:32.009952 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064408c-5562-4a10-a8b6-e412ca66698d None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:32.010568 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064408c-5562-4a10-a8b6-e412ca66698d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:32.010568 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064408c-5562-4a10-a8b6-e412ca66698d None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:32.011945 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 728/1447] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 10:20:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:20:32.018377 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af05885d-adaf-4dca-8cf0-116c0ca71cc7 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:32.018837 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af05885d-adaf-4dca-8cf0-116c0ca71cc7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:32.018983 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af05885d-adaf-4dca-8cf0-116c0ca71cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:32.079821 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:20:32.100495 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-af05885d-adaf-4dca-8cf0-116c0ca71cc7 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:20:32.101693 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-af05885d-adaf-4dca-8cf0-116c0ca71cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:32.202014 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 720/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 10:20:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:20:32.245400 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0064408c-5562-4a10-a8b6-e412ca66698d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:20:32.246952 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0064408c-5562-4a10-a8b6-e412ca66698d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:32.364511 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0064408c-5562-4a10-a8b6-e412ca66698d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:20:32.416386 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:20:32.453550 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd90a623-8ae3-41fa-a27b-3243f88a3ba9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:20:32.454175 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd90a623-8ae3-41fa-a27b-3243f88a3ba9 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:32.454452 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd90a623-8ae3-41fa-a27b-3243f88a3ba9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:32.454718 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd90a623-8ae3-41fa-a27b-3243f88a3ba9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:32.456386 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fd90a623-8ae3-41fa-a27b-3243f88a3ba9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:32.585802 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd90a623-8ae3-41fa-a27b-3243f88a3ba9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:20:32.587287 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd90a623-8ae3-41fa-a27b-3243f88a3ba9 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:20:32.606785 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 729/1449] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:20:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:20:34.793296 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe426c-54c8-4dbe-8ecb-6193631d8ca1 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:34.793910 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe426c-54c8-4dbe-8ecb-6193631d8ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:34.793910 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe426c-54c8-4dbe-8ecb-6193631d8ca1 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:34.795526 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 721/1450] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 10:20:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:20:34.801874 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc006622-afc2-4d94-9a2d-d4910eea9adb None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:34.802591 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc006622-afc2-4d94-9a2d-d4910eea9adb None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:34.802773 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc006622-afc2-4d94-9a2d-d4910eea9adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:34.874920 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-bc006622-afc2-4d94-9a2d-d4910eea9adb None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:20:34.876168 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bc006622-afc2-4d94-9a2d-d4910eea9adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:34.898834 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:20:34.982386 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 730/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 10:20:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:20:35.026178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe426c-54c8-4dbe-8ecb-6193631d8ca1 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:20:35.027775 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-14fe426c-54c8-4dbe-8ecb-6193631d8ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:35.137771 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe426c-54c8-4dbe-8ecb-6193631d8ca1 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:20:35.158488 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:20:35.216425 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b5527c-8545-42e7-a471-7875bc8d4137 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:20:35.217068 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b5527c-8545-42e7-a471-7875bc8d4137 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:35.217341 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b5527c-8545-42e7-a471-7875bc8d4137 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:35.217605 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b5527c-8545-42e7-a471-7875bc8d4137 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:35.219261 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f0b5527c-8545-42e7-a471-7875bc8d4137 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:35.345283 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b5527c-8545-42e7-a471-7875bc8d4137 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:20:35.346704 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b5527c-8545-42e7-a471-7875bc8d4137 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:20:35.365601 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 722/1452] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:20:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:20:37.504484 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3179ca85-418c-4e66-bda4-14ea72595828 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:37.505378 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3179ca85-418c-4e66-bda4-14ea72595828 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:37.505378 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3179ca85-418c-4e66-bda4-14ea72595828 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:37.507598 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 731/1453] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 10:20:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:20:37.513176 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc51e56-60f2-4c27-b198-2689989311c8 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:37.514158 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc51e56-60f2-4c27-b198-2689989311c8 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:37.514158 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc51e56-60f2-4c27-b198-2689989311c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:37.579962 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:20:37.582821 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-adc51e56-60f2-4c27-b198-2689989311c8 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:20:37.584671 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-adc51e56-60f2-4c27-b198-2689989311c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:37.686485 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 723/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 10:20:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:20:37.727497 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3179ca85-418c-4e66-bda4-14ea72595828 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:20:37.729077 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3179ca85-418c-4e66-bda4-14ea72595828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:37.852138 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3179ca85-418c-4e66-bda4-14ea72595828 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:20:37.911039 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:20:37.937027 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859808ac-547e-42ee-b07c-6b13900d06dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:20:37.937648 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859808ac-547e-42ee-b07c-6b13900d06dd service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:37.937927 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859808ac-547e-42ee-b07c-6b13900d06dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:37.938197 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859808ac-547e-42ee-b07c-6b13900d06dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:37.939868 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-859808ac-547e-42ee-b07c-6b13900d06dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:38.054726 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859808ac-547e-42ee-b07c-6b13900d06dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:20:38.056206 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859808ac-547e-42ee-b07c-6b13900d06dd service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:20:38.074395 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 732/1455] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:20:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:20:40.125599 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a94d4-43d9-4d1e-8e04-42108b0c5fdf None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:40.126167 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a94d4-43d9-4d1e-8e04-42108b0c5fdf None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:40.126167 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a94d4-43d9-4d1e-8e04-42108b0c5fdf None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:40.128661 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 724/1456] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 10:20:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:20:40.135364 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b74d6-b2a2-4583-9e1e-cffd73bfe5e7 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:40.135767 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b74d6-b2a2-4583-9e1e-cffd73bfe5e7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:40.135969 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b74d6-b2a2-4583-9e1e-cffd73bfe5e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:40.205765 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a58b74d6-b2a2-4583-9e1e-cffd73bfe5e7 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:20:40.207074 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a58b74d6-b2a2-4583-9e1e-cffd73bfe5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:40.236611 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:20:40.305369 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 733/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 10:20:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:20:40.350861 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a94d4-43d9-4d1e-8e04-42108b0c5fdf None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:20:40.353394 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-762a94d4-43d9-4d1e-8e04-42108b0c5fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:40.473071 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a94d4-43d9-4d1e-8e04-42108b0c5fdf None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:20:40.502889 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:20:40.553311 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84098a88-5448-4bdb-ab90-3fad2ada8a60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:20:40.553934 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84098a88-5448-4bdb-ab90-3fad2ada8a60 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:40.554209 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84098a88-5448-4bdb-ab90-3fad2ada8a60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:40.554466 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84098a88-5448-4bdb-ab90-3fad2ada8a60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:40.556141 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-84098a88-5448-4bdb-ab90-3fad2ada8a60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:40.669166 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84098a88-5448-4bdb-ab90-3fad2ada8a60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:20:40.670571 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84098a88-5448-4bdb-ab90-3fad2ada8a60 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:20:40.689077 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 725/1458] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:20:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:20:42.752481 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6839be6-f653-41bb-b84f-f320e3cef5ee None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:42.753396 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6839be6-f653-41bb-b84f-f320e3cef5ee None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:42.753396 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6839be6-f653-41bb-b84f-f320e3cef5ee None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:42.755664 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 734/1459] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 10:20:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:20:42.763915 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f488d2-ed37-4572-8452-cb62586456d8 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:42.764523 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f488d2-ed37-4572-8452-cb62586456d8 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:42.764641 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f488d2-ed37-4572-8452-cb62586456d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:42.853236 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-72f488d2-ed37-4572-8452-cb62586456d8 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:20:42.854507 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-72f488d2-ed37-4572-8452-cb62586456d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:42.863600 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:20:42.963516 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 726/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 10:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:20:43.003661 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6839be6-f653-41bb-b84f-f320e3cef5ee None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:20:43.005801 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a6839be6-f653-41bb-b84f-f320e3cef5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:43.131661 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6839be6-f653-41bb-b84f-f320e3cef5ee None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:20:43.193775 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:20:43.215536 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a4376-e796-4737-9780-c1dadc6ed302 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:20:43.216170 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a4376-e796-4737-9780-c1dadc6ed302 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:43.216455 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a4376-e796-4737-9780-c1dadc6ed302 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:43.216734 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a4376-e796-4737-9780-c1dadc6ed302 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:43.218385 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-124a4376-e796-4737-9780-c1dadc6ed302 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:43.346409 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a4376-e796-4737-9780-c1dadc6ed302 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:20:43.347811 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a4376-e796-4737-9780-c1dadc6ed302 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:20:43.366198 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 735/1461] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:20:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:20:45.389676 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f897ca44-b35d-43d3-b419-58444dcbec43 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:45.390211 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f897ca44-b35d-43d3-b419-58444dcbec43 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:45.390211 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f897ca44-b35d-43d3-b419-58444dcbec43 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:45.391636 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 727/1462] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 10:20:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:20:45.398060 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8613ad-ffb0-4f34-bf7f-f9c5b0cead9f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:45.398764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8613ad-ffb0-4f34-bf7f-f9c5b0cead9f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:45.398998 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8613ad-ffb0-4f34-bf7f-f9c5b0cead9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:45.476929 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-4c8613ad-ffb0-4f34-bf7f-f9c5b0cead9f None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:20:45.478149 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4c8613ad-ffb0-4f34-bf7f-f9c5b0cead9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:45.517686 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:20:45.577541 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 736/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 10:20:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:20:45.618445 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f897ca44-b35d-43d3-b419-58444dcbec43 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:20:45.620985 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f897ca44-b35d-43d3-b419-58444dcbec43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:45.739665 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f897ca44-b35d-43d3-b419-58444dcbec43 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:20:45.781349 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:20:45.820807 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54645a32-e9de-45f5-a458-a8e32b991b64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:20:45.821422 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54645a32-e9de-45f5-a458-a8e32b991b64 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:45.821696 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54645a32-e9de-45f5-a458-a8e32b991b64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:45.821958 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54645a32-e9de-45f5-a458-a8e32b991b64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:45.823604 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-54645a32-e9de-45f5-a458-a8e32b991b64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:45.933738 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54645a32-e9de-45f5-a458-a8e32b991b64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:20:45.935285 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54645a32-e9de-45f5-a458-a8e32b991b64 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:20:45.953644 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 728/1464] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:20:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:20:48.002071 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f758f-bd4b-4589-abba-4fe421a8e8d0 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:48.002679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f758f-bd4b-4589-abba-4fe421a8e8d0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:48.002679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f758f-bd4b-4589-abba-4fe421a8e8d0 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:48.004036 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 737/1465] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 10:20:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:20:48.010393 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd390b-68f9-44bd-89b2-f54c846b9741 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:48.010846 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd390b-68f9-44bd-89b2-f54c846b9741 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:48.010986 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd390b-68f9-44bd-89b2-f54c846b9741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:48.098326 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-65fd390b-68f9-44bd-89b2-f54c846b9741 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:20:48.099569 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-65fd390b-68f9-44bd-89b2-f54c846b9741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:48.127193 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:20:48.210671 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 729/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 10:20:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:20:48.256280 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705f758f-bd4b-4589-abba-4fe421a8e8d0 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:20:48.258738 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-705f758f-bd4b-4589-abba-4fe421a8e8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:48.377273 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705f758f-bd4b-4589-abba-4fe421a8e8d0 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:20:48.447701 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:20:48.454937 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517dc824-fd97-45a3-b589-75bf5e83efd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:20:48.455604 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517dc824-fd97-45a3-b589-75bf5e83efd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:48.455881 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517dc824-fd97-45a3-b589-75bf5e83efd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:48.456144 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517dc824-fd97-45a3-b589-75bf5e83efd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:48.457751 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-517dc824-fd97-45a3-b589-75bf5e83efd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:48.576663 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517dc824-fd97-45a3-b589-75bf5e83efd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:20:48.578110 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517dc824-fd97-45a3-b589-75bf5e83efd1 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:20:48.597847 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 738/1467] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:20:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:20:50.609196 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0d3d8-79bf-42e3-b3f4-1bdffdb284cf None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:50.610287 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0d3d8-79bf-42e3-b3f4-1bdffdb284cf None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:50.610287 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0d3d8-79bf-42e3-b3f4-1bdffdb284cf None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:50.612653 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 730/1468] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 10:20:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:20:50.618321 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b01d9-7517-4a64-ac0b-38275aa96c6c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:50.618969 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b01d9-7517-4a64-ac0b-38275aa96c6c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:50.618969 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b01d9-7517-4a64-ac0b-38275aa96c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:50.691100 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-373b01d9-7517-4a64-ac0b-38275aa96c6c None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:20:50.692295 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-373b01d9-7517-4a64-ac0b-38275aa96c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:50.740607 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:20:50.791859 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 739/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 10:20:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:20:50.829218 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0d3d8-79bf-42e3-b3f4-1bdffdb284cf None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:20:50.830788 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-dac0d3d8-79bf-42e3-b3f4-1bdffdb284cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:50.948000 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0d3d8-79bf-42e3-b3f4-1bdffdb284cf None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:20:50.999321 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:20:51.028002 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430812d0-f3cb-4846-ac51-36672b1f9e62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:20:51.028626 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430812d0-f3cb-4846-ac51-36672b1f9e62 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:20:51.028936 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430812d0-f3cb-4846-ac51-36672b1f9e62 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:20:51.029159 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430812d0-f3cb-4846-ac51-36672b1f9e62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:20:51.030863 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-430812d0-f3cb-4846-ac51-36672b1f9e62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:20:51.151747 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430812d0-f3cb-4846-ac51-36672b1f9e62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:20:51.153355 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430812d0-f3cb-4846-ac51-36672b1f9e62 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:20:51.173133 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 731/1470] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:20:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:21:00.789782 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=22 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:21:01.170617 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=66 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:21:10.745331 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f4eff-1064-458e-a071-3336c2f00f3b None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:21:10.746253 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f4eff-1064-458e-a071-3336c2f00f3b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:21:10.746253 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f4eff-1064-458e-a071-3336c2f00f3b None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:21:10.748519 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 740/1471] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:21:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:21:10.754183 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3282aea-cbe5-456f-b05f-67991f648cfd None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:21:10.754639 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3282aea-cbe5-456f-b05f-67991f648cfd None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:21:10.754746 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3282aea-cbe5-456f-b05f-67991f648cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:21:10.829398 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c3282aea-cbe5-456f-b05f-67991f648cfd None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:21:10.830815 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c3282aea-cbe5-456f-b05f-67991f648cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:21:10.935357 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 732/1472] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:21:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:21:11.028524 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c4eb7-31d0-4a20-a623-99ebc83f4b85 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:21:11.028808 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c4eb7-31d0-4a20-a623-99ebc83f4b85 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:21:11.029083 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c4eb7-31d0-4a20-a623-99ebc83f4b85 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:21:11.030503 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 741/1473] 10.222.0.40 () {54 vars in 966 bytes} [Fri Aug 5 10:21:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:21:11.036086 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863a640-40db-4b7b-ac29-c0444d52788c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:21:11.036411 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863a640-40db-4b7b-ac29-c0444d52788c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:21:11.036684 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863a640-40db-4b7b-ac29-c0444d52788c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:21:11.116326 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-9863a640-40db-4b7b-ac29-c0444d52788c None None] MFA Rules not processed for user `ac0a8cf7f8b74e88a04444ec38547528`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:21:11.117548 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9863a640-40db-4b7b-ac29-c0444d52788c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:21:11.140638 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:21:11.200781 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 733/1474] 10.222.0.40 () {58 vars in 1063 bytes} [Fri Aug 5 10:21:11 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:21:11.211994 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c4eb7-31d0-4a20-a623-99ebc83f4b85 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:21:11.213568 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-729c4eb7-31d0-4a20-a623-99ebc83f4b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:21:11.321339 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c4eb7-31d0-4a20-a623-99ebc83f4b85 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:21:11.403843 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7acef9-7220-45d7-8d30-4722e6aae8b5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac0a8cf7f8b74e88a04444ec38547528', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:21:11.404472 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7acef9-7220-45d7-8d30-4722e6aae8b5 service glance] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:21:11.404820 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7acef9-7220-45d7-8d30-4722e6aae8b5 service glance] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:21:11.405092 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7acef9-7220-45d7-8d30-4722e6aae8b5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:21:11.406735 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9d7acef9-7220-45d7-8d30-4722e6aae8b5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:21:11.464266 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:21:11.526595 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7acef9-7220-45d7-8d30-4722e6aae8b5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:21:11.528448 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7acef9-7220-45d7-8d30-4722e6aae8b5 service glance] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:21:11.547811 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 742/1475] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:21:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:21:13.648161 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf04d50-7cb5-44eb-bc78-8b93b8cfee99 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:21:13.648732 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf04d50-7cb5-44eb-bc78-8b93b8cfee99 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:21:13.648732 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf04d50-7cb5-44eb-bc78-8b93b8cfee99 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:21:13.650067 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 734/1476] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:21:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:21:13.656800 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe18faf-295c-472a-a62d-e162b9eae20d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:21:13.657544 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe18faf-295c-472a-a62d-e162b9eae20d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:21:13.657717 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe18faf-295c-472a-a62d-e162b9eae20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:21:13.730404 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-3fe18faf-295c-472a-a62d-e162b9eae20d None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:21:13.731627 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3fe18faf-295c-472a-a62d-e162b9eae20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:21:13.841533 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 743/1477] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:21:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:21:13.872007 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf04d50-7cb5-44eb-bc78-8b93b8cfee99 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:21:13.873531 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-cdf04d50-7cb5-44eb-bc78-8b93b8cfee99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:21:14.013093 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf04d50-7cb5-44eb-bc78-8b93b8cfee99 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:21:14.068172 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:21:14.117500 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0be0013-9654-40c5-8cab-a75df882cd66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:21:14.118156 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be0013-9654-40c5-8cab-a75df882cd66 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:21:14.118432 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be0013-9654-40c5-8cab-a75df882cd66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:21:14.118704 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be0013-9654-40c5-8cab-a75df882cd66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:21:14.119428 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0be0013-9654-40c5-8cab-a75df882cd66 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:21:14.121177 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: WARNING py.warnings [None req-a0be0013-9654-40c5-8cab-a75df882cd66 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 10:21:14.121177 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: warnings.warn(msg) Aug 05 10:21:14.121177 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: Aug 05 10:21:14.122319 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0be0013-9654-40c5-8cab-a75df882cd66 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:21:14.143697 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 735/1478] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 10:21:13 2022] GET /identity/v3/projects => generated 2553 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:21:16.201758 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb5ea7-71fb-4399-ab2c-eaa3bfb244ed None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:21:16.202271 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb5ea7-71fb-4399-ab2c-eaa3bfb244ed None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:21:16.202271 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb5ea7-71fb-4399-ab2c-eaa3bfb244ed None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:21:16.203713 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 744/1479] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:21:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:21:16.209580 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d40f4-932b-4da9-b3c6-29dbd52b6f92 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:21:16.210062 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d40f4-932b-4da9-b3c6-29dbd52b6f92 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:21:16.210155 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d40f4-932b-4da9-b3c6-29dbd52b6f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:21:16.280079 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b78d40f4-932b-4da9-b3c6-29dbd52b6f92 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:21:16.281289 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b78d40f4-932b-4da9-b3c6-29dbd52b6f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:21:16.388018 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 736/1480] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:21:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:21:16.435457 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eb5ea7-71fb-4399-ab2c-eaa3bfb244ed None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:21:16.437005 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a2eb5ea7-71fb-4399-ab2c-eaa3bfb244ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:21:16.447368 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:21:16.604095 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eb5ea7-71fb-4399-ab2c-eaa3bfb244ed None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:21:16.700399 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:21:16.734383 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40066c-9d92-4689-bd0c-223aa0e0974a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:21:16.735041 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40066c-9d92-4689-bd0c-223aa0e0974a service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:21:16.735319 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40066c-9d92-4689-bd0c-223aa0e0974a service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:21:16.735585 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40066c-9d92-4689-bd0c-223aa0e0974a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:21:16.737537 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9a40066c-9d92-4689-bd0c-223aa0e0974a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:21:16.840113 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a40066c-9d92-4689-bd0c-223aa0e0974a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:21:16.842559 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a40066c-9d92-4689-bd0c-223aa0e0974a service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:21:16.865233 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 745/1481] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:21:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:21:19.088664 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd34b45-2593-4bd9-ae0e-8f592b793626 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:21:19.089249 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd34b45-2593-4bd9-ae0e-8f592b793626 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:21:19.089249 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd34b45-2593-4bd9-ae0e-8f592b793626 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:21:19.091491 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 737/1482] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:21:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:21:19.099219 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f50abd-d8a2-40c6-84e8-e220185690c6 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:21:19.099696 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f50abd-d8a2-40c6-84e8-e220185690c6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:21:19.099788 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f50abd-d8a2-40c6-84e8-e220185690c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:21:19.168401 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f8f50abd-d8a2-40c6-84e8-e220185690c6 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:21:19.169620 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f8f50abd-d8a2-40c6-84e8-e220185690c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:21:19.218477 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:21:19.272679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 746/1483] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:21:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:21:19.313003 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd34b45-2593-4bd9-ae0e-8f592b793626 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:21:19.315417 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ecd34b45-2593-4bd9-ae0e-8f592b793626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:21:19.351890 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:21:19.441123 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd34b45-2593-4bd9-ae0e-8f592b793626 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:21:19.522219 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a13b0-24be-456b-ad33-261ae6544cd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:21:19.522854 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a13b0-24be-456b-ad33-261ae6544cd6 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:21:19.523167 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a13b0-24be-456b-ad33-261ae6544cd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:21:19.523442 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a13b0-24be-456b-ad33-261ae6544cd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:21:19.525090 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-708a13b0-24be-456b-ad33-261ae6544cd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:21:19.612415 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:21:19.652906 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708a13b0-24be-456b-ad33-261ae6544cd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:21:19.654358 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708a13b0-24be-456b-ad33-261ae6544cd6 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:21:19.674161 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 738/1484] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:21:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:21:21.817456 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0c019-8be4-49db-84e5-8175fe41631f None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:21:21.818135 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0c019-8be4-49db-84e5-8175fe41631f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:21:21.818135 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0c019-8be4-49db-84e5-8175fe41631f None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:21:21.820074 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 747/1485] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:21:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:21:21.828737 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bfbf55-4a12-4ab4-b512-b62d67e1d680 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:21:21.829156 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bfbf55-4a12-4ab4-b512-b62d67e1d680 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:21:21.829269 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bfbf55-4a12-4ab4-b512-b62d67e1d680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:21:21.904960 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-e6bfbf55-4a12-4ab4-b512-b62d67e1d680 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:21:21.906338 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e6bfbf55-4a12-4ab4-b512-b62d67e1d680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:21:22.010385 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 739/1486] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:21:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:21:22.049006 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca0c019-8be4-49db-84e5-8175fe41631f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:21:22.050596 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fca0c019-8be4-49db-84e5-8175fe41631f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:21:22.157062 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca0c019-8be4-49db-84e5-8175fe41631f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:21:22.228239 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:21:22.230085 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d302bb3-5e88-431d-a64e-ea6f4db836d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:21:22.230738 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d302bb3-5e88-431d-a64e-ea6f4db836d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:21:22.231028 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d302bb3-5e88-431d-a64e-ea6f4db836d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:21:22.231313 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d302bb3-5e88-431d-a64e-ea6f4db836d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:21:22.233779 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9d302bb3-5e88-431d-a64e-ea6f4db836d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:21:22.347669 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d302bb3-5e88-431d-a64e-ea6f4db836d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:21:22.349145 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d302bb3-5e88-431d-a64e-ea6f4db836d1 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:21:22.368396 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 748/1487] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:21:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:21:24.454777 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d86aa3-5217-430d-af69-9193948ab0a6 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:21:24.455361 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d86aa3-5217-430d-af69-9193948ab0a6 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:21:24.455361 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d86aa3-5217-430d-af69-9193948ab0a6 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:21:24.456734 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 740/1488] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:21:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:21:24.462222 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd511d5-e63b-49fc-999a-556246d8e4ee None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:21:24.462681 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd511d5-e63b-49fc-999a-556246d8e4ee None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:21:24.462792 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd511d5-e63b-49fc-999a-556246d8e4ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:21:24.529851 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8fd511d5-e63b-49fc-999a-556246d8e4ee None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:21:24.531072 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8fd511d5-e63b-49fc-999a-556246d8e4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:21:24.591080 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:21:24.638250 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 749/1489] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:21:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:21:24.680103 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d86aa3-5217-430d-af69-9193948ab0a6 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:21:24.681642 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f1d86aa3-5217-430d-af69-9193948ab0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:21:24.720993 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:21:24.801166 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d86aa3-5217-430d-af69-9193948ab0a6 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:21:24.881027 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f9e881-b2d3-41e8-96b5-2a52bba3f8f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:21:24.881659 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9e881-b2d3-41e8-96b5-2a52bba3f8f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:21:24.881935 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9e881-b2d3-41e8-96b5-2a52bba3f8f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:21:24.882200 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9e881-b2d3-41e8-96b5-2a52bba3f8f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:21:24.883892 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-76f9e881-b2d3-41e8-96b5-2a52bba3f8f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:21:24.975388 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:21:25.006242 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f9e881-b2d3-41e8-96b5-2a52bba3f8f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:21:25.007791 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f9e881-b2d3-41e8-96b5-2a52bba3f8f5 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:21:25.026679 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 741/1490] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:21:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:21:27.326720 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfbf6f-db0f-4e0a-a49f-c9fe74afc699 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:21:27.327360 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfbf6f-db0f-4e0a-a49f-c9fe74afc699 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:21:27.327360 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfbf6f-db0f-4e0a-a49f-c9fe74afc699 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:21:27.328674 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 750/1491] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 10:21:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:21:27.334169 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ddb34-6627-4a5f-b8ea-06fbe5ee7b13 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:21:27.334595 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ddb34-6627-4a5f-b8ea-06fbe5ee7b13 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:21:27.334720 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ddb34-6627-4a5f-b8ea-06fbe5ee7b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:21:27.405821 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-4c3ddb34-6627-4a5f-b8ea-06fbe5ee7b13 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:21:27.407055 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4c3ddb34-6627-4a5f-b8ea-06fbe5ee7b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:21:27.507815 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 742/1492] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 10:21:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:21:27.546965 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cfbf6f-db0f-4e0a-a49f-c9fe74afc699 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:21:27.548493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d3cfbf6f-db0f-4e0a-a49f-c9fe74afc699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:21:27.673133 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cfbf6f-db0f-4e0a-a49f-c9fe74afc699 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:21:27.747943 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42f71d8-ed51-4e5d-8642-b6063b60690e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:21:27.748568 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f71d8-ed51-4e5d-8642-b6063b60690e service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:21:27.748847 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f71d8-ed51-4e5d-8642-b6063b60690e service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:21:27.749124 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f71d8-ed51-4e5d-8642-b6063b60690e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:21:27.750750 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f42f71d8-ed51-4e5d-8642-b6063b60690e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:21:27.760573 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:21:27.873587 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42f71d8-ed51-4e5d-8642-b6063b60690e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:21:27.875072 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42f71d8-ed51-4e5d-8642-b6063b60690e service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:21:27.894860 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 751/1493] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:21:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:21:37.505633 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=82 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:21:37.892240 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=50 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:22:38.652265 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c256be-515e-495a-b357-084ab890aab1 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:22:38.652895 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c256be-515e-495a-b357-084ab890aab1 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:22:38.652895 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c256be-515e-495a-b357-084ab890aab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:22:38.731563 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-51c256be-515e-495a-b357-084ab890aab1 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:22:38.732791 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-51c256be-515e-495a-b357-084ab890aab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:22:38.833900 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 743/1494] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:22:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:22:38.842614 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c77cff-b129-4dec-841f-0191c20027f8 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:22:38.843180 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c77cff-b129-4dec-841f-0191c20027f8 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:22:38.843180 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c77cff-b129-4dec-841f-0191c20027f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:22:38.920311 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-86c77cff-b129-4dec-841f-0191c20027f8 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:22:38.921517 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-86c77cff-b129-4dec-841f-0191c20027f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:22:39.043904 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 752/1495] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:22:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:22:39.049036 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c256be-515e-495a-b357-084ab890aab1 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:22:39.050299 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-51c256be-515e-495a-b357-084ab890aab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:22:39.062936 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c256be-515e-495a-b357-084ab890aab1 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:22:39.128527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:22:39.173513 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2113abc-2b63-456d-8e56-54fbca22cea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:22:39.174148 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2113abc-2b63-456d-8e56-54fbca22cea3 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:22:39.174422 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2113abc-2b63-456d-8e56-54fbca22cea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:22:39.174697 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2113abc-2b63-456d-8e56-54fbca22cea3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:22:39.175428 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2113abc-2b63-456d-8e56-54fbca22cea3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:22:39.175760 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2113abc-2b63-456d-8e56-54fbca22cea3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:22:39.176824 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: WARNING py.warnings [None req-c2113abc-2b63-456d-8e56-54fbca22cea3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 05 10:22:39.176824 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: warnings.warn(msg) Aug 05 10:22:39.176824 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: Aug 05 10:22:39.177707 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2113abc-2b63-456d-8e56-54fbca22cea3 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:22:39.189662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 744/1496] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:22:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:22:39.198352 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c77cff-b129-4dec-841f-0191c20027f8 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:22:39.208158 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c77cff-b129-4dec-841f-0191c20027f8 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:22:39.274605 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:22:39.315831 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a90da-21c5-4e75-bd05-3e37e5f962e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:22:39.316384 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a90da-21c5-4e75-bd05-3e37e5f962e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:22:39.316664 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a90da-21c5-4e75-bd05-3e37e5f962e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:22:39.316935 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a90da-21c5-4e75-bd05-3e37e5f962e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:22:39.317594 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09a90da-21c5-4e75-bd05-3e37e5f962e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:22:39.319442 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: WARNING py.warnings [None req-e09a90da-21c5-4e75-bd05-3e37e5f962e2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 10:22:39.319442 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: warnings.warn(msg) Aug 05 10:22:39.319442 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: Aug 05 10:22:39.320757 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09a90da-21c5-4e75-bd05-3e37e5f962e2 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:22:39.384790 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 753/1497] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:22:39 2022] POST /identity/v3/projects => generated 377 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:22:39.389299 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2113abc-2b63-456d-8e56-54fbca22cea3 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:22:39.390834 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c2113abc-2b63-456d-8e56-54fbca22cea3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:22:39.425556 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2113abc-2b63-456d-8e56-54fbca22cea3 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:22:39.521764 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2efee90-b78d-4528-901e-f1d41e49fdda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:22:39.522358 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2efee90-b78d-4528-901e-f1d41e49fdda admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:22:39.522668 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2efee90-b78d-4528-901e-f1d41e49fdda admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:22:39.522941 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2efee90-b78d-4528-901e-f1d41e49fdda admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:22:39.523642 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2efee90-b78d-4528-901e-f1d41e49fdda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:22:39.524668 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: WARNING py.warnings [None req-c2efee90-b78d-4528-901e-f1d41e49fdda admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 10:22:39.524668 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: warnings.warn(msg) Aug 05 10:22:39.524668 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: Aug 05 10:22:39.525884 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2efee90-b78d-4528-901e-f1d41e49fdda admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:22:39.572410 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 745/1498] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:22:39 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:22:39.577320 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a90da-21c5-4e75-bd05-3e37e5f962e2 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:22:39.579709 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e09a90da-21c5-4e75-bd05-3e37e5f962e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:22:39.652025 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a90da-21c5-4e75-bd05-3e37e5f962e2 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:22:39.737405 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:22:39.750224 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0facdff1-0234-493d-89d1-c00d332742f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:22:39.750847 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0facdff1-0234-493d-89d1-c00d332742f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:22:39.751151 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0facdff1-0234-493d-89d1-c00d332742f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:22:39.751418 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0facdff1-0234-493d-89d1-c00d332742f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:22:39.752089 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0facdff1-0234-493d-89d1-c00d332742f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:22:39.753113 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: WARNING py.warnings [None req-0facdff1-0234-493d-89d1-c00d332742f6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 05 10:22:39.753113 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: warnings.warn(msg) Aug 05 10:22:39.753113 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: Aug 05 10:22:39.753993 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0facdff1-0234-493d-89d1-c00d332742f6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:22:39.767012 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 754/1499] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:22:39 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 10:22:39.771325 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2efee90-b78d-4528-901e-f1d41e49fdda admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:22:39.773118 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c2efee90-b78d-4528-901e-f1d41e49fdda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:22:39.785334 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2efee90-b78d-4528-901e-f1d41e49fdda admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:22:39.815716 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:22:39.896523 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de69b0b-27a0-4534-a06c-89016a66ae31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:22:39.897065 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de69b0b-27a0-4534-a06c-89016a66ae31 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:22:39.897343 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de69b0b-27a0-4534-a06c-89016a66ae31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:22:39.897614 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de69b0b-27a0-4534-a06c-89016a66ae31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:22:39.898382 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de69b0b-27a0-4534-a06c-89016a66ae31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:22:39.899442 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: WARNING py.warnings [None req-9de69b0b-27a0-4534-a06c-89016a66ae31 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 05 10:22:39.899442 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: warnings.warn(msg) Aug 05 10:22:39.899442 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: Aug 05 10:22:39.900334 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de69b0b-27a0-4534-a06c-89016a66ae31 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:22:39.914559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 746/1500] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:22:39 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:22:39.921470 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0facdff1-0234-493d-89d1-c00d332742f6 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:22:39.923821 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0facdff1-0234-493d-89d1-c00d332742f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:22:39.964008 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0facdff1-0234-493d-89d1-c00d332742f6 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:22:40.071432 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9817d-955a-450a-ab68-b6c0c52c24ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:22:40.072101 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9817d-955a-450a-ab68-b6c0c52c24ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:22:40.072379 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9817d-955a-450a-ab68-b6c0c52c24ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:22:40.072648 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9817d-955a-450a-ab68-b6c0c52c24ab admin admin] PATH_INFO: `/v3/projects/633d23dc3e4b408e9c4be36ef6a3e910/users/9ec9fa2424dc401591d386eac2d64193/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:22:40.095441 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da9817d-955a-450a-ab68-b6c0c52c24ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=633d23dc3e4b408e9c4be36ef6a3e910, user_id=9ec9fa2424dc401591d386eac2d64193, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:22:40.096478 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: WARNING py.warnings [None req-3da9817d-955a-450a-ab68-b6c0c52c24ab admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 10:22:40.096478 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: warnings.warn(msg) Aug 05 10:22:40.096478 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: Aug 05 10:22:40.098396 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da9817d-955a-450a-ab68-b6c0c52c24ab admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:22:40.125319 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 755/1501] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:22:39 2022] PUT /identity/v3/projects/633d23dc3e4b408e9c4be36ef6a3e910/users/9ec9fa2424dc401591d386eac2d64193/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:22:40.137798 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8ee17-494e-4ea4-82c0-87b0e11a89a2 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:22:40.138093 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8ee17-494e-4ea4-82c0-87b0e11a89a2 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:22:40.138368 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8ee17-494e-4ea4-82c0-87b0e11a89a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:22:40.199181 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-17e8ee17-494e-4ea4-82c0-87b0e11a89a2 None None] MFA Rules not processed for user `9ec9fa2424dc401591d386eac2d64193`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:22:40.200456 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-17e8ee17-494e-4ea4-82c0-87b0e11a89a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:22:40.285178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 747/1502] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:22:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:22:40.300565 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978def86-c2e9-46cd-b3bf-6e5a4d76c258 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:22:40.300849 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978def86-c2e9-46cd-b3bf-6e5a4d76c258 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:22:40.301095 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978def86-c2e9-46cd-b3bf-6e5a4d76c258 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:22:40.302463 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 756/1503] 10.222.0.40 () {54 vars in 964 bytes} [Fri Aug 5 10:22:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:22:40.307426 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94839c24-3984-479c-9732-d46b140d7dd9 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:22:40.307743 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94839c24-3984-479c-9732-d46b140d7dd9 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:22:40.308011 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94839c24-3984-479c-9732-d46b140d7dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:22:40.340718 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:22:40.388727 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-94839c24-3984-479c-9732-d46b140d7dd9 None None] MFA Rules not processed for user `f70369fd5b334d888de0c091d8885eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:22:40.390519 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-94839c24-3984-479c-9732-d46b140d7dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:22:40.479080 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 748/1504] 10.222.0.40 () {58 vars in 1061 bytes} [Fri Aug 5 10:22:40 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:22:40.487248 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978def86-c2e9-46cd-b3bf-6e5a4d76c258 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:22:40.488776 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-978def86-c2e9-46cd-b3bf-6e5a4d76c258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:22:40.538644 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:22:40.592893 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978def86-c2e9-46cd-b3bf-6e5a4d76c258 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:22:40.675601 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011a8e75-2c76-46c9-aeba-59354f03dd4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:22:40.676217 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a8e75-2c76-46c9-aeba-59354f03dd4a service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:22:40.676492 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a8e75-2c76-46c9-aeba-59354f03dd4a service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:22:40.676763 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a8e75-2c76-46c9-aeba-59354f03dd4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:22:40.678373 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-011a8e75-2c76-46c9-aeba-59354f03dd4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:22:40.803725 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011a8e75-2c76-46c9-aeba-59354f03dd4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:22:40.805740 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011a8e75-2c76-46c9-aeba-59354f03dd4a service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:22:40.809297 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:22:40.825335 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 757/1505] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:22:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:22:40.879083 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa33d61e-258d-48e0-8be9-d95baf3dcbd4 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:22:40.879536 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa33d61e-258d-48e0-8be9-d95baf3dcbd4 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:22:40.879637 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa33d61e-258d-48e0-8be9-d95baf3dcbd4 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:22:40.880959 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 749/1506] 10.222.0.40 () {52 vars in 830 bytes} [Fri Aug 5 10:22:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:22:40.885367 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011a8e75-2c76-46c9-aeba-59354f03dd4a service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:22:40.886582 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-011a8e75-2c76-46c9-aeba-59354f03dd4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:22:40.994059 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011a8e75-2c76-46c9-aeba-59354f03dd4a service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:22:41.100194 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8b7ff-d904-42a1-8350-b9fe0a35dbc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:22:41.100744 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8b7ff-d904-42a1-8350-b9fe0a35dbc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:22:41.101020 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8b7ff-d904-42a1-8350-b9fe0a35dbc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:22:41.101432 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8b7ff-d904-42a1-8350-b9fe0a35dbc6 admin admin] PATH_INFO: `/v3/users/9ec9fa2424dc401591d386eac2d64193` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:22:41.114783 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f8b7ff-d904-42a1-8350-b9fe0a35dbc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ec9fa2424dc401591d386eac2d64193)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:22:41.115848 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: WARNING py.warnings [None req-11f8b7ff-d904-42a1-8350-b9fe0a35dbc6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 10:22:41.115848 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: warnings.warn(msg) Aug 05 10:22:41.115848 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: Aug 05 10:22:41.117087 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f8b7ff-d904-42a1-8350-b9fe0a35dbc6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:22:41.128629 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:22:41.191897 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-11f8b7ff-d904-42a1-8350-b9fe0a35dbc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ec9fa2424dc401591d386eac2d64193'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:22:41.192339 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-11f8b7ff-d904-42a1-8350-b9fe0a35dbc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ec9fa2424dc401591d386eac2d64193'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:22:41.192847 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-11f8b7ff-d904-42a1-8350-b9fe0a35dbc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ec9fa2424dc401591d386eac2d64193'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:22:41.205673 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-11f8b7ff-d904-42a1-8350-b9fe0a35dbc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ec9fa2424dc401591d386eac2d64193'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:22:41.222349 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-11f8b7ff-d904-42a1-8350-b9fe0a35dbc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ec9fa2424dc401591d386eac2d64193'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:22:41.261988 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 758/1507] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:22:40 2022] DELETE /identity/v3/users/9ec9fa2424dc401591d386eac2d64193 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:22:41.268969 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f15da-e0c7-4745-949e-ceb0f1658f31 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:22:41.269403 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f15da-e0c7-4745-949e-ceb0f1658f31 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:22:41.269793 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f15da-e0c7-4745-949e-ceb0f1658f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:22:41.342921 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-8b2f15da-e0c7-4745-949e-ceb0f1658f31 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:22:41.344753 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8b2f15da-e0c7-4745-949e-ceb0f1658f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:22:41.431686 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:22:41.451525 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 750/1508] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:22:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:22:41.461541 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8b7ff-d904-42a1-8350-b9fe0a35dbc6 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:22:41.462755 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-11f8b7ff-d904-42a1-8350-b9fe0a35dbc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:22:41.571291 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8b7ff-d904-42a1-8350-b9fe0a35dbc6 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:22:41.629665 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:22:41.652179 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca58d8-d16f-482a-b85b-9054a335659b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:22:41.652804 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca58d8-d16f-482a-b85b-9054a335659b service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:22:41.653084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca58d8-d16f-482a-b85b-9054a335659b service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:22:41.653359 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca58d8-d16f-482a-b85b-9054a335659b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:22:41.655055 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-89ca58d8-d16f-482a-b85b-9054a335659b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:22:41.770358 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ca58d8-d16f-482a-b85b-9054a335659b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:22:41.771857 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ca58d8-d16f-482a-b85b-9054a335659b service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:22:41.792318 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 759/1509] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:22:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 10:22:42.169006 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f15da-e0c7-4745-949e-ceb0f1658f31 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:22:42.170283 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8b2f15da-e0c7-4745-949e-ceb0f1658f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:22:42.182375 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f15da-e0c7-4745-949e-ceb0f1658f31 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:22:42.277347 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22216b2-78a4-4310-a5aa-771cfbd40531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:22:42.278007 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22216b2-78a4-4310-a5aa-771cfbd40531 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:22:42.278291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22216b2-78a4-4310-a5aa-771cfbd40531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:22:42.278562 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22216b2-78a4-4310-a5aa-771cfbd40531 admin admin] PATH_INFO: `/v3/projects/633d23dc3e4b408e9c4be36ef6a3e910` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:22:42.289928 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22216b2-78a4-4310-a5aa-771cfbd40531 admin admin] RBAC: Authorizing `identity:delete_project(project_id=633d23dc3e4b408e9c4be36ef6a3e910)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:22:42.291016 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: WARNING py.warnings [None req-f22216b2-78a4-4310-a5aa-771cfbd40531 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 10:22:42.291016 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: warnings.warn(msg) Aug 05 10:22:42.291016 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: Aug 05 10:22:42.292290 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22216b2-78a4-4310-a5aa-771cfbd40531 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:22:42.357899 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f22216b2-78a4-4310-a5aa-771cfbd40531 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '633d23dc3e4b408e9c4be36ef6a3e910', 'request_id': 'req-f22216b2-78a4-4310-a5aa-771cfbd40531'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:22:42.365381 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f22216b2-78a4-4310-a5aa-771cfbd40531 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '633d23dc3e4b408e9c4be36ef6a3e910', 'request_id': 'req-f22216b2-78a4-4310-a5aa-771cfbd40531'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:22:42.377316 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f22216b2-78a4-4310-a5aa-771cfbd40531 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '633d23dc3e4b408e9c4be36ef6a3e910', 'request_id': 'req-f22216b2-78a4-4310-a5aa-771cfbd40531'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:22:42.416063 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f22216b2-78a4-4310-a5aa-771cfbd40531 admin admin] The token cache is being invalidate because project 633d23dc3e4b408e9c4be36ef6a3e910 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:22:42.416382 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f22216b2-78a4-4310-a5aa-771cfbd40531 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:22:42.416929 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f22216b2-78a4-4310-a5aa-771cfbd40531 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:22:42.418421 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 751/1510] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:22:42 2022] DELETE /identity/v3/projects/633d23dc3e4b408e9c4be36ef6a3e910 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:22:51.789847 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=62 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:22:52.413711 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:24:09.456704 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f51cc-9a01-4f47-97cc-1ac58d9f5a31 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:24:09.457321 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f51cc-9a01-4f47-97cc-1ac58d9f5a31 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:24:09.457321 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f51cc-9a01-4f47-97cc-1ac58d9f5a31 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:24:09.459677 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 760/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 10:24:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:24:09.464947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d848da-60d9-4194-a499-07c608ca2615 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:24:09.465426 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d848da-60d9-4194-a499-07c608ca2615 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:24:09.465528 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d848da-60d9-4194-a499-07c608ca2615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:24:09.550836 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-61d848da-60d9-4194-a499-07c608ca2615 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:24:09.552101 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-61d848da-60d9-4194-a499-07c608ca2615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:24:09.672042 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 752/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 10:24:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:24:09.824730 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637f51cc-9a01-4f47-97cc-1ac58d9f5a31 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:24:09.828673 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-637f51cc-9a01-4f47-97cc-1ac58d9f5a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:24:09.960107 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637f51cc-9a01-4f47-97cc-1ac58d9f5a31 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:24:09.967819 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b963d6-38ee-4773-b895-d7f89097722b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:24:09.968750 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b963d6-38ee-4773-b895-d7f89097722b service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:24:09.969085 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b963d6-38ee-4773-b895-d7f89097722b service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:24:09.969516 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b963d6-38ee-4773-b895-d7f89097722b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:24:09.983206 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b963d6-38ee-4773-b895-d7f89097722b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:24:09.985573 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b963d6-38ee-4773-b895-d7f89097722b service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:24:09.997056 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 761/1513] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:24:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:24:10.836671 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469fa84-235d-41dc-978f-1c7206288ca5 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:24:10.837254 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469fa84-235d-41dc-978f-1c7206288ca5 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:24:10.837254 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469fa84-235d-41dc-978f-1c7206288ca5 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:24:10.838515 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 753/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 10:24:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:24:10.844164 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c99933b-3172-49f9-87e3-e9f52459e1b7 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:24:10.844615 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c99933b-3172-49f9-87e3-e9f52459e1b7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:24:10.844718 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c99933b-3172-49f9-87e3-e9f52459e1b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:24:10.873173 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6c99933b-3172-49f9-87e3-e9f52459e1b7 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:24:10.874516 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6c99933b-3172-49f9-87e3-e9f52459e1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:24:10.883463 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 762/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 10:24:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:24:11.035638 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5469fa84-235d-41dc-978f-1c7206288ca5 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:24:11.051491 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5469fa84-235d-41dc-978f-1c7206288ca5 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:24:11.058657 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3115e805-9fb3-4cbf-b1d1-1eb9e74063bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:24:11.059293 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3115e805-9fb3-4cbf-b1d1-1eb9e74063bf service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:24:11.059571 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3115e805-9fb3-4cbf-b1d1-1eb9e74063bf service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:24:11.059833 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3115e805-9fb3-4cbf-b1d1-1eb9e74063bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:24:11.069522 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3115e805-9fb3-4cbf-b1d1-1eb9e74063bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:24:11.071223 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3115e805-9fb3-4cbf-b1d1-1eb9e74063bf service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:24:11.080896 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 754/1516] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:24:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:24:11.421930 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56f97c-4858-402c-8ff7-21125b210c37 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:24:11.422158 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56f97c-4858-402c-8ff7-21125b210c37 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:24:11.422375 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56f97c-4858-402c-8ff7-21125b210c37 None None] PATH_INFO: `/v3` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:24:11.423765 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 763/1517] 10.222.0.36 () {54 vars in 925 bytes} [Fri Aug 5 10:24:11 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:24:11.430655 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578df355-41b4-47de-b288-56d32e4f9a9b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:24:11.430966 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578df355-41b4-47de-b288-56d32e4f9a9b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:24:11.431246 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578df355-41b4-47de-b288-56d32e4f9a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:24:11.495114 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-578df355-41b4-47de-b288-56d32e4f9a9b None None] MFA Rules not processed for user `8743f5bb0be54d57a957617636b0b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:24:11.496316 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-578df355-41b4-47de-b288-56d32e4f9a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:24:11.514710 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:24:11.571326 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 755/1518] 10.222.0.36 () {58 vars in 1015 bytes} [Fri Aug 5 10:24:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:24:20.867662 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=56 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:24:20.909459 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969ea95-5eb9-46c8-a3d7-c6ebc34bda12 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:24:20.909801 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969ea95-5eb9-46c8-a3d7-c6ebc34bda12 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:24:20.910135 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969ea95-5eb9-46c8-a3d7-c6ebc34bda12 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:24:20.912048 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 764/1519] 10.222.0.40 () {54 vars in 961 bytes} [Fri Aug 5 10:24:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:24:20.917694 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345aada-b36a-4311-91ed-257202b0bb39 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:24:20.918267 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345aada-b36a-4311-91ed-257202b0bb39 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:24:20.918267 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345aada-b36a-4311-91ed-257202b0bb39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:24:20.940218 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-0345aada-b36a-4311-91ed-257202b0bb39 None None] MFA Rules not processed for user `8743f5bb0be54d57a957617636b0b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:24:20.941433 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0345aada-b36a-4311-91ed-257202b0bb39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:24:20.955696 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 756/1520] 10.222.0.40 () {58 vars in 1058 bytes} [Fri Aug 5 10:24:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:24:20.964859 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9969ea95-5eb9-46c8-a3d7-c6ebc34bda12 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:24:20.982892 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9969ea95-5eb9-46c8-a3d7-c6ebc34bda12 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:24:20.988228 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32985293-9340-417f-aee0-48f9ff167a20 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8743f5bb0be54d57a957617636b0b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:24:20.988847 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32985293-9340-417f-aee0-48f9ff167a20 service placement] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:24:20.989127 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32985293-9340-417f-aee0-48f9ff167a20 service placement] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:24:20.989402 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32985293-9340-417f-aee0-48f9ff167a20 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:24:21.001660 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32985293-9340-417f-aee0-48f9ff167a20 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:24:21.003298 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32985293-9340-417f-aee0-48f9ff167a20 service placement] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:24:21.014227 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 765/1521] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:24:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:24:22.311586 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d2cd01-2308-4ff6-83ad-cd1097e08d0e None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:24:22.312152 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d2cd01-2308-4ff6-83ad-cd1097e08d0e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:24:22.312152 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d2cd01-2308-4ff6-83ad-cd1097e08d0e None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:24:22.313514 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 757/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 10:24:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:24:22.319661 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d8e14b-eb31-47c5-9870-074e36f2be42 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:24:22.320438 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d8e14b-eb31-47c5-9870-074e36f2be42 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:24:22.320595 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d8e14b-eb31-47c5-9870-074e36f2be42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:24:22.352343 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-10d8e14b-eb31-47c5-9870-074e36f2be42 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:24:22.354337 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-10d8e14b-eb31-47c5-9870-074e36f2be42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:24:22.363960 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 766/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 10:24:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:24:22.521174 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d2cd01-2308-4ff6-83ad-cd1097e08d0e None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:24:22.538119 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d2cd01-2308-4ff6-83ad-cd1097e08d0e None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:24:22.544436 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db7a64-db57-4267-918a-a13033d16431 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:24:22.545052 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db7a64-db57-4267-918a-a13033d16431 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:24:22.545334 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db7a64-db57-4267-918a-a13033d16431 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:24:22.545606 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db7a64-db57-4267-918a-a13033d16431 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:24:22.553924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90db7a64-db57-4267-918a-a13033d16431 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:24:22.555562 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90db7a64-db57-4267-918a-a13033d16431 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:24:22.565012 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 758/1524] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:24:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:24:23.370345 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0687a93-42c8-4aad-8d53-2f0d43389e87 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:24:23.370849 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0687a93-42c8-4aad-8d53-2f0d43389e87 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:24:23.370849 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0687a93-42c8-4aad-8d53-2f0d43389e87 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:24:23.372329 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 767/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 10:24:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:24:23.377368 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b1965-3691-42ec-98ef-a77cd1efc5f5 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:24:23.377838 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b1965-3691-42ec-98ef-a77cd1efc5f5 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:24:23.377964 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b1965-3691-42ec-98ef-a77cd1efc5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:24:23.399830 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-8e5b1965-3691-42ec-98ef-a77cd1efc5f5 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:24:23.401634 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8e5b1965-3691-42ec-98ef-a77cd1efc5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:24:23.410271 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 759/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 10:24:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:24:23.558939 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0687a93-42c8-4aad-8d53-2f0d43389e87 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:24:23.574991 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0687a93-42c8-4aad-8d53-2f0d43389e87 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:24:23.581350 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997668a3-5ac4-4d71-a1fa-48bcedf2cb8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:24:23.581951 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997668a3-5ac4-4d71-a1fa-48bcedf2cb8d service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:24:23.582244 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997668a3-5ac4-4d71-a1fa-48bcedf2cb8d service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:24:23.582491 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997668a3-5ac4-4d71-a1fa-48bcedf2cb8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:24:23.591121 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997668a3-5ac4-4d71-a1fa-48bcedf2cb8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:24:23.592706 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997668a3-5ac4-4d71-a1fa-48bcedf2cb8d service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:24:23.602123 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 768/1527] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:24:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:24:27.397773 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb942179-3e02-4c22-b568-6492a43add97 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:24:27.398331 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb942179-3e02-4c22-b568-6492a43add97 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:24:27.398331 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb942179-3e02-4c22-b568-6492a43add97 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:24:27.399673 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 760/1528] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 10:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:24:27.406338 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45a076-f9f0-4f10-a71e-c38b5a16211e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:24:27.406798 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45a076-f9f0-4f10-a71e-c38b5a16211e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:24:27.406928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45a076-f9f0-4f10-a71e-c38b5a16211e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:24:27.431798 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8e45a076-f9f0-4f10-a71e-c38b5a16211e None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:24:27.432950 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8e45a076-f9f0-4f10-a71e-c38b5a16211e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:24:27.442234 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 769/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 10:24:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:24:27.519298 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb942179-3e02-4c22-b568-6492a43add97 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:24:27.521261 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-bb942179-3e02-4c22-b568-6492a43add97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:24:27.609581 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb942179-3e02-4c22-b568-6492a43add97 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:24:27.613868 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6366556-110a-4022-8fc2-b34b70e970f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:24:27.614531 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6366556-110a-4022-8fc2-b34b70e970f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:24:27.614817 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6366556-110a-4022-8fc2-b34b70e970f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:24:27.615138 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6366556-110a-4022-8fc2-b34b70e970f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:24:27.625444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6366556-110a-4022-8fc2-b34b70e970f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:24:27.626884 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6366556-110a-4022-8fc2-b34b70e970f5 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:24:27.636974 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 761/1530] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:24:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:24:29.657982 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce78672-0c14-47af-9b3d-5a93fa05d1a1 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:24:29.658938 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce78672-0c14-47af-9b3d-5a93fa05d1a1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:24:29.658938 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce78672-0c14-47af-9b3d-5a93fa05d1a1 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:24:29.661101 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 770/1531] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 10:24:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:24:29.667181 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7cfda-e88a-4dd9-b567-8abd00af8c27 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:24:29.667610 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7cfda-e88a-4dd9-b567-8abd00af8c27 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:24:29.667724 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7cfda-e88a-4dd9-b567-8abd00af8c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:24:29.688634 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b9d7cfda-e88a-4dd9-b567-8abd00af8c27 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:24:29.689827 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b9d7cfda-e88a-4dd9-b567-8abd00af8c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:24:29.698495 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 762/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 10:24:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:24:29.737676 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce78672-0c14-47af-9b3d-5a93fa05d1a1 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:24:29.758472 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce78672-0c14-47af-9b3d-5a93fa05d1a1 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:24:29.763956 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9144c4-0114-4dce-b7d0-bc654fdfa196 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:24:29.764566 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9144c4-0114-4dce-b7d0-bc654fdfa196 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:24:29.764838 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9144c4-0114-4dce-b7d0-bc654fdfa196 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:24:29.765105 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9144c4-0114-4dce-b7d0-bc654fdfa196 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:24:29.775236 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9144c4-0114-4dce-b7d0-bc654fdfa196 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:24:29.776726 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9144c4-0114-4dce-b7d0-bc654fdfa196 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:24:29.787006 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 771/1533] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:24:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:24:33.283191 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf843c3a-c547-46de-a60d-5b3383557f31 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:24:33.284044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf843c3a-c547-46de-a60d-5b3383557f31 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:24:33.284044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf843c3a-c547-46de-a60d-5b3383557f31 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:24:33.286330 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 763/1534] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 10:24:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:24:33.293892 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d101a9-79db-499d-9d25-dfb0eb8986da None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:24:33.294332 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d101a9-79db-499d-9d25-dfb0eb8986da None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:24:33.294438 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d101a9-79db-499d-9d25-dfb0eb8986da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:24:33.315434 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-80d101a9-79db-499d-9d25-dfb0eb8986da None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:24:33.316644 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-80d101a9-79db-499d-9d25-dfb0eb8986da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:24:33.325913 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 772/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 10:24:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:24:33.630652 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf843c3a-c547-46de-a60d-5b3383557f31 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:24:33.633856 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-bf843c3a-c547-46de-a60d-5b3383557f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:24:33.714884 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:24:33.734574 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf843c3a-c547-46de-a60d-5b3383557f31 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:24:33.741249 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ce3a9-d539-427b-ac80-c5b20fa5b845 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:24:33.742079 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ce3a9-d539-427b-ac80-c5b20fa5b845 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:24:33.742434 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ce3a9-d539-427b-ac80-c5b20fa5b845 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:24:33.742734 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ce3a9-d539-427b-ac80-c5b20fa5b845 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:24:33.753672 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0ce3a9-d539-427b-ac80-c5b20fa5b845 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:24:33.755150 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0ce3a9-d539-427b-ac80-c5b20fa5b845 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:24:33.765004 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 764/1536] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:24:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:24:35.802107 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f19d2-2813-4a9a-b8b6-b0c7b93bb160 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:24:35.802724 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f19d2-2813-4a9a-b8b6-b0c7b93bb160 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:24:35.802724 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f19d2-2813-4a9a-b8b6-b0c7b93bb160 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:24:35.804112 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 773/1537] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 10:24:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:24:35.810712 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce001a-36e9-4602-a02e-25481f754a88 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:24:35.811196 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce001a-36e9-4602-a02e-25481f754a88 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:24:35.811366 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce001a-36e9-4602-a02e-25481f754a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:24:35.838335 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-2bce001a-36e9-4602-a02e-25481f754a88 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:24:35.839584 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2bce001a-36e9-4602-a02e-25481f754a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:24:35.848129 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 765/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 10:24:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:24:36.162013 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f19d2-2813-4a9a-b8b6-b0c7b93bb160 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:24:36.165361 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2a2f19d2-2813-4a9a-b8b6-b0c7b93bb160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:24:36.180725 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f19d2-2813-4a9a-b8b6-b0c7b93bb160 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:24:36.184761 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb44ab-e74b-4c13-ac14-8d85baa05f3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:24:36.185381 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb44ab-e74b-4c13-ac14-8d85baa05f3a service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:24:36.185660 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb44ab-e74b-4c13-ac14-8d85baa05f3a service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:24:36.185932 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb44ab-e74b-4c13-ac14-8d85baa05f3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:24:36.195395 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb44ab-e74b-4c13-ac14-8d85baa05f3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:24:36.196960 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb44ab-e74b-4c13-ac14-8d85baa05f3a service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:24:36.206633 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 774/1539] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:24:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:24:45.832255 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=11 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:24:46.202766 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=33 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:01.089641 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49067e73-06c5-40e7-a2e5-66e112755e9b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:01.091169 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49067e73-06c5-40e7-a2e5-66e112755e9b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:01.091677 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49067e73-06c5-40e7-a2e5-66e112755e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:01.116665 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-49067e73-06c5-40e7-a2e5-66e112755e9b None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:01.117802 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-49067e73-06c5-40e7-a2e5-66e112755e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:01.130290 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 766/1540] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:26:01.140340 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f02c15e-f180-496f-a7f0-4dd4747cd9aa None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:01.141546 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f02c15e-f180-496f-a7f0-4dd4747cd9aa None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:01.142318 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f02c15e-f180-496f-a7f0-4dd4747cd9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:01.165780 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43868406-652d-4d7f-8c2e-8917ba77620a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:01.166068 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43868406-652d-4d7f-8c2e-8917ba77620a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:01.166346 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43868406-652d-4d7f-8c2e-8917ba77620a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:01.169410 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-2f02c15e-f180-496f-a7f0-4dd4747cd9aa None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:01.170631 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2f02c15e-f180-496f-a7f0-4dd4747cd9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:01.184454 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 775/1541] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:01.192024 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f02c15e-f180-496f-a7f0-4dd4747cd9aa None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:01.202831 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f02c15e-f180-496f-a7f0-4dd4747cd9aa None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:01.209116 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-43868406-652d-4d7f-8c2e-8917ba77620a None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:01.210363 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-43868406-652d-4d7f-8c2e-8917ba77620a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:01.211859 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaada9e9-b2ef-409f-80c0-f8a4c703fcbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:01.212723 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaada9e9-b2ef-409f-80c0-f8a4c703fcbb admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:01.213170 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaada9e9-b2ef-409f-80c0-f8a4c703fcbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:01.213622 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaada9e9-b2ef-409f-80c0-f8a4c703fcbb admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:01.214532 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaada9e9-b2ef-409f-80c0-f8a4c703fcbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:26:01.215041 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaada9e9-b2ef-409f-80c0-f8a4c703fcbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:01.216267 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: WARNING py.warnings [None req-eaada9e9-b2ef-409f-80c0-f8a4c703fcbb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 05 10:26:01.216267 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: warnings.warn(msg) Aug 05 10:26:01.216267 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: Aug 05 10:26:01.217532 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaada9e9-b2ef-409f-80c0-f8a4c703fcbb admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:01.227450 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 767/1542] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:26:01.229952 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092937db-c52e-4d7c-86a2-33db15e14c62 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:01.230235 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092937db-c52e-4d7c-86a2-33db15e14c62 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:01.230511 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092937db-c52e-4d7c-86a2-33db15e14c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:01.240431 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 776/1543] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:26:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:26:01.251027 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaada9e9-b2ef-409f-80c0-f8a4c703fcbb admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:01.267435 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaada9e9-b2ef-409f-80c0-f8a4c703fcbb admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:01.270090 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-092937db-c52e-4d7c-86a2-33db15e14c62 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:01.273962 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-092937db-c52e-4d7c-86a2-33db15e14c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:01.277286 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd403028-296c-4dc8-a822-a43c65b4b69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:01.278114 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd403028-296c-4dc8-a822-a43c65b4b69a admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:01.278564 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd403028-296c-4dc8-a822-a43c65b4b69a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:01.279021 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd403028-296c-4dc8-a822-a43c65b4b69a admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:01.279870 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd403028-296c-4dc8-a822-a43c65b4b69a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:01.281939 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd403028-296c-4dc8-a822-a43c65b4b69a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:01.284925 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 768/1544] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:26:01.289958 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092937db-c52e-4d7c-86a2-33db15e14c62 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:01.306993 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092937db-c52e-4d7c-86a2-33db15e14c62 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:01.309863 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02be933d-1103-4736-a06a-f739ed14f610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:01.310514 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02be933d-1103-4736-a06a-f739ed14f610 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:01.310792 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02be933d-1103-4736-a06a-f739ed14f610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:01.315521 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02be933d-1103-4736-a06a-f739ed14f610 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:01.316357 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02be933d-1103-4736-a06a-f739ed14f610 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:26:01.316671 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02be933d-1103-4736-a06a-f739ed14f610 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:01.318312 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02be933d-1103-4736-a06a-f739ed14f610 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:01.339979 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 769/1545] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:26:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:26:01.355375 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02be933d-1103-4736-a06a-f739ed14f610 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:01.366884 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 777/1546] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:26:01 2022] POST /identity/v3/projects => generated 379 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:26:01.375087 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02be933d-1103-4736-a06a-f739ed14f610 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:01.378207 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd403028-296c-4dc8-a822-a43c65b4b69a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:01.487314 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9311b9f1-fb62-493c-8a3e-bcbe30960e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:01.488014 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311b9f1-fb62-493c-8a3e-bcbe30960e01 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:01.488296 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311b9f1-fb62-493c-8a3e-bcbe30960e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:01.488569 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311b9f1-fb62-493c-8a3e-bcbe30960e01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:01.489479 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9311b9f1-fb62-493c-8a3e-bcbe30960e01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:01.490799 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: WARNING py.warnings [None req-9311b9f1-fb62-493c-8a3e-bcbe30960e01 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 10:26:01.490799 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: warnings.warn(msg) Aug 05 10:26:01.490799 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: Aug 05 10:26:01.492012 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9311b9f1-fb62-493c-8a3e-bcbe30960e01 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:01.498100 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd403028-296c-4dc8-a822-a43c65b4b69a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:01.510136 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94b591c-2976-4cb3-8b07-09ebeb83d20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:01.510776 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b591c-2976-4cb3-8b07-09ebeb83d20b admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:01.511079 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b591c-2976-4cb3-8b07-09ebeb83d20b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:01.511346 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b591c-2976-4cb3-8b07-09ebeb83d20b admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:01.512052 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94b591c-2976-4cb3-8b07-09ebeb83d20b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:01.513086 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: WARNING py.warnings [None req-b94b591c-2976-4cb3-8b07-09ebeb83d20b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 10:26:01.513086 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: warnings.warn(msg) Aug 05 10:26:01.513086 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: Aug 05 10:26:01.514285 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94b591c-2976-4cb3-8b07-09ebeb83d20b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:01.546019 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 778/1547] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:26:01 2022] POST /identity/v3/users => generated 290 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:26:01.547448 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b01c53-2e6e-4cd3-bc31-59829012ae9d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:01.547742 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b01c53-2e6e-4cd3-bc31-59829012ae9d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:01.548017 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b01c53-2e6e-4cd3-bc31-59829012ae9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:01.551385 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 770/1548] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:26:01 2022] POST /identity/v3/projects => generated 391 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:26:01.552781 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b5b54b-9d95-4e9a-88b2-b9f31eaa5a19 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:01.553067 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b5b54b-9d95-4e9a-88b2-b9f31eaa5a19 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:01.553341 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b5b54b-9d95-4e9a-88b2-b9f31eaa5a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:01.569296 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f2b01c53-2e6e-4cd3-bc31-59829012ae9d None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:01.570507 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f2b01c53-2e6e-4cd3-bc31-59829012ae9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:01.575023 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-12b5b54b-9d95-4e9a-88b2-b9f31eaa5a19 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:01.576286 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-12b5b54b-9d95-4e9a-88b2-b9f31eaa5a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:01.634683 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 779/1549] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:01.638766 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466ee5a-b336-4260-997a-61123ce75cfa None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:01.639085 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466ee5a-b336-4260-997a-61123ce75cfa None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:01.639359 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466ee5a-b336-4260-997a-61123ce75cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:01.640232 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 771/1550] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:01.641477 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea6ab2-1d8e-46b6-84e4-f08444545a6d None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:01.641770 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea6ab2-1d8e-46b6-84e4-f08444545a6d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:01.642050 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea6ab2-1d8e-46b6-84e4-f08444545a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:01.662141 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c3ea6ab2-1d8e-46b6-84e4-f08444545a6d None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:01.662402 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b466ee5a-b336-4260-997a-61123ce75cfa None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:01.663364 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b466ee5a-b336-4260-997a-61123ce75cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:01.663511 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c3ea6ab2-1d8e-46b6-84e4-f08444545a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:01.678558 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 780/1551] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:01.678770 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 772/1552] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:01.678770 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea6ab2-1d8e-46b6-84e4-f08444545a6d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:01.678991 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9860bd0-e155-4a73-9ae9-c97562d6ac31 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:01.678991 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9860bd0-e155-4a73-9ae9-c97562d6ac31 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:01.678991 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9860bd0-e155-4a73-9ae9-c97562d6ac31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:01.689305 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea6ab2-1d8e-46b6-84e4-f08444545a6d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:01.691997 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e9860bd0-e155-4a73-9ae9-c97562d6ac31 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:01.693188 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e9860bd0-e155-4a73-9ae9-c97562d6ac31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:01.694395 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83395ed2-7fe4-4094-94c2-6e137b25a4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:01.695052 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83395ed2-7fe4-4094-94c2-6e137b25a4a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:01.695331 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83395ed2-7fe4-4094-94c2-6e137b25a4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:01.695599 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83395ed2-7fe4-4094-94c2-6e137b25a4a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:01.696295 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83395ed2-7fe4-4094-94c2-6e137b25a4a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:01.697891 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83395ed2-7fe4-4094-94c2-6e137b25a4a5 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:01.701125 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 781/1553] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:01.701900 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9860bd0-e155-4a73-9ae9-c97562d6ac31 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:01.711736 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 773/1554] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:01.711736 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302af28a-351c-4f9e-b9ce-fcea96545286 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:01.711736 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302af28a-351c-4f9e-b9ce-fcea96545286 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:01.711736 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302af28a-351c-4f9e-b9ce-fcea96545286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:01.714800 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9860bd0-e155-4a73-9ae9-c97562d6ac31 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:01.724453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a61d9e-332c-4228-911c-f6f157e9e8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:01.725093 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a61d9e-332c-4228-911c-f6f157e9e8d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:01.725832 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a61d9e-332c-4228-911c-f6f157e9e8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:01.725832 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a61d9e-332c-4228-911c-f6f157e9e8d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:01.726299 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a61d9e-332c-4228-911c-f6f157e9e8d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:01.728270 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a61d9e-332c-4228-911c-f6f157e9e8d7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:01.731053 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-302af28a-351c-4f9e-b9ce-fcea96545286 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:01.732754 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-302af28a-351c-4f9e-b9ce-fcea96545286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:01.741678 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 774/1555] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:01.742925 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70076d32-34a3-448d-8801-b2cb35dfe2b8 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:01.743213 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70076d32-34a3-448d-8801-b2cb35dfe2b8 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:01.743502 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70076d32-34a3-448d-8801-b2cb35dfe2b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:01.763605 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 782/1556] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:26:01 2022] POST /identity/v3/users => generated 296 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:26:01.763605 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfdd287-c1a1-4f9b-8d04-b3d26997d540 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:01.763605 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfdd287-c1a1-4f9b-8d04-b3d26997d540 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:01.763605 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfdd287-c1a1-4f9b-8d04-b3d26997d540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:01.765035 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-70076d32-34a3-448d-8801-b2cb35dfe2b8 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:01.766227 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-70076d32-34a3-448d-8801-b2cb35dfe2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:01.774752 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 775/1557] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:01.775974 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd5991-18f4-467a-bd83-594024672626 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:01.776221 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd5991-18f4-467a-bd83-594024672626 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:01.776488 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd5991-18f4-467a-bd83-594024672626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:01.787857 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1cfdd287-c1a1-4f9b-8d04-b3d26997d540 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:01.789067 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1cfdd287-c1a1-4f9b-8d04-b3d26997d540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:01.795882 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-e5fd5991-18f4-467a-bd83-594024672626 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:01.797023 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e5fd5991-18f4-467a-bd83-594024672626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:01.804110 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 783/1558] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:01.804110 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a1d927-f940-470c-b7ec-0ddebfa76b5e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:01.804110 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a1d927-f940-470c-b7ec-0ddebfa76b5e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:01.804110 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a1d927-f940-470c-b7ec-0ddebfa76b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:01.808980 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 776/1559] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:01.810173 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e722993-977d-4d6a-a8fb-643a3889f4b4 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:01.810449 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e722993-977d-4d6a-a8fb-643a3889f4b4 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:01.810715 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e722993-977d-4d6a-a8fb-643a3889f4b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:01.818202 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:01.831929 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b3a1d927-f940-470c-b7ec-0ddebfa76b5e None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:01.832106 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-5e722993-977d-4d6a-a8fb-643a3889f4b4 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:01.833117 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5e722993-977d-4d6a-a8fb-643a3889f4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:01.833244 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b3a1d927-f940-470c-b7ec-0ddebfa76b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:01.841136 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 777/1560] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:01.845343 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e722993-977d-4d6a-a8fb-643a3889f4b4 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:01.845442 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 784/1561] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:01.845442 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a00f86-a040-4729-b747-b8a2ccdd493f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:01.845442 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a00f86-a040-4729-b747-b8a2ccdd493f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:01.845442 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a00f86-a040-4729-b747-b8a2ccdd493f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:01.855693 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e722993-977d-4d6a-a8fb-643a3889f4b4 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:01.860829 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f25576-4ecf-4c57-b0b6-ffb15c066a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:01.861575 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f25576-4ecf-4c57-b0b6-ffb15c066a18 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:01.862036 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f25576-4ecf-4c57-b0b6-ffb15c066a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:01.862775 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f25576-4ecf-4c57-b0b6-ffb15c066a18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:01.863524 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f25576-4ecf-4c57-b0b6-ffb15c066a18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:01.864192 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e4a00f86-a040-4729-b747-b8a2ccdd493f None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:01.865205 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f25576-4ecf-4c57-b0b6-ffb15c066a18 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:01.865401 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e4a00f86-a040-4729-b747-b8a2ccdd493f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:01.867475 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:01.874488 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 785/1562] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:01.875311 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a00f86-a040-4729-b747-b8a2ccdd493f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:01.880317 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 778/1563] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:01.882507 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f25576-4ecf-4c57-b0b6-ffb15c066a18 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:01.886572 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a00f86-a040-4729-b747-b8a2ccdd493f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:01.891347 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e955c1b6-bcbd-474d-a0ea-fb4f445f9ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:01.891959 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e955c1b6-bcbd-474d-a0ea-fb4f445f9ed3 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:01.892238 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e955c1b6-bcbd-474d-a0ea-fb4f445f9ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:01.892344 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f25576-4ecf-4c57-b0b6-ffb15c066a18 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:01.892512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e955c1b6-bcbd-474d-a0ea-fb4f445f9ed3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:01.893470 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e955c1b6-bcbd-474d-a0ea-fb4f445f9ed3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:01.894867 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e955c1b6-bcbd-474d-a0ea-fb4f445f9ed3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:01.897071 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f0de08-eef1-4bfb-857f-7fd5e35c1cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:01.897670 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0de08-eef1-4bfb-857f-7fd5e35c1cf7 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:01.897943 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0de08-eef1-4bfb-857f-7fd5e35c1cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:01.898587 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0de08-eef1-4bfb-857f-7fd5e35c1cf7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:01.899307 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f0de08-eef1-4bfb-857f-7fd5e35c1cf7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:26:01.899608 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f0de08-eef1-4bfb-857f-7fd5e35c1cf7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:01.901188 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f0de08-eef1-4bfb-857f-7fd5e35c1cf7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:01.904965 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 786/1564] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:01.905744 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e955c1b6-bcbd-474d-a0ea-fb4f445f9ed3 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:01.913995 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 779/1565] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:26:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:26:01.914710 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f0de08-eef1-4bfb-857f-7fd5e35c1cf7 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:01.917276 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e955c1b6-bcbd-474d-a0ea-fb4f445f9ed3 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:01.922098 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a84a104-a623-4998-9064-8d32bfa0603c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:01.922712 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a84a104-a623-4998-9064-8d32bfa0603c admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:01.923021 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a84a104-a623-4998-9064-8d32bfa0603c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:01.923292 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a84a104-a623-4998-9064-8d32bfa0603c admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:01.924159 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a84a104-a623-4998-9064-8d32bfa0603c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:26:01.924455 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f0de08-eef1-4bfb-857f-7fd5e35c1cf7 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:01.925064 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a84a104-a623-4998-9064-8d32bfa0603c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:01.926672 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a84a104-a623-4998-9064-8d32bfa0603c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:01.929496 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09c34b4-1e38-4dae-8103-78b3a73abe6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:01.929654 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09c34b4-1e38-4dae-8103-78b3a73abe6a admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:01.929904 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09c34b4-1e38-4dae-8103-78b3a73abe6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:01.930168 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09c34b4-1e38-4dae-8103-78b3a73abe6a admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:01.930845 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09c34b4-1e38-4dae-8103-78b3a73abe6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:26:01.931179 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09c34b4-1e38-4dae-8103-78b3a73abe6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:01.932889 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09c34b4-1e38-4dae-8103-78b3a73abe6a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:01.940769 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 787/1566] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:26:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:26:01.941775 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a84a104-a623-4998-9064-8d32bfa0603c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:01.951549 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 780/1567] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:26:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:26:01.951717 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a84a104-a623-4998-9064-8d32bfa0603c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:01.952253 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09c34b4-1e38-4dae-8103-78b3a73abe6a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:01.956800 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f2b6e-3a0c-4b50-9eb3-2adaca8c7201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:01.957424 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f2b6e-3a0c-4b50-9eb3-2adaca8c7201 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:01.957701 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f2b6e-3a0c-4b50-9eb3-2adaca8c7201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:01.957968 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f2b6e-3a0c-4b50-9eb3-2adaca8c7201 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:01.958652 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37f2b6e-3a0c-4b50-9eb3-2adaca8c7201 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:26:01.958984 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37f2b6e-3a0c-4b50-9eb3-2adaca8c7201 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:01.960586 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37f2b6e-3a0c-4b50-9eb3-2adaca8c7201 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:01.964225 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09c34b4-1e38-4dae-8103-78b3a73abe6a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:01.968937 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0867372b-d1c7-43bd-8b91-91c04d8f0182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:01.969545 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867372b-d1c7-43bd-8b91-91c04d8f0182 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:01.969825 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867372b-d1c7-43bd-8b91-91c04d8f0182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:01.970089 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867372b-d1c7-43bd-8b91-91c04d8f0182 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:01.971026 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0867372b-d1c7-43bd-8b91-91c04d8f0182 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:26:01.971448 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0867372b-d1c7-43bd-8b91-91c04d8f0182 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:01.973065 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0867372b-d1c7-43bd-8b91-91c04d8f0182 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:01.973424 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 788/1568] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:26:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:26:01.974168 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f2b6e-3a0c-4b50-9eb3-2adaca8c7201 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:01.984085 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f2b6e-3a0c-4b50-9eb3-2adaca8c7201 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:01.986436 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 781/1569] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:26:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:26:01.987317 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0867372b-d1c7-43bd-8b91-91c04d8f0182 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:01.989679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe953e-8cd8-4d83-a23d-f7498ceea27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:01.990297 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe953e-8cd8-4d83-a23d-f7498ceea27c admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:01.990573 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe953e-8cd8-4d83-a23d-f7498ceea27c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:01.990843 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe953e-8cd8-4d83-a23d-f7498ceea27c admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:01.991563 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfe953e-8cd8-4d83-a23d-f7498ceea27c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:26:01.991865 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfe953e-8cd8-4d83-a23d-f7498ceea27c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:01.993585 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfe953e-8cd8-4d83-a23d-f7498ceea27c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:01.997251 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0867372b-d1c7-43bd-8b91-91c04d8f0182 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:02.002407 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5fbaa0-e598-46d8-a7a9-a2cc4ccd8f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:02.003075 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5fbaa0-e598-46d8-a7a9-a2cc4ccd8f04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:02.003284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5fbaa0-e598-46d8-a7a9-a2cc4ccd8f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:02.003561 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5fbaa0-e598-46d8-a7a9-a2cc4ccd8f04 admin admin] PATH_INFO: `/v3/projects/5a7a56c0fd0c43d4803634f30479187a/users/9ace48889c484e4fb5fed0f8f80c3288/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:02.006540 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 789/1570] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:26:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:26:02.014446 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe953e-8cd8-4d83-a23d-f7498ceea27c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:02.019409 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5fbaa0-e598-46d8-a7a9-a2cc4ccd8f04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a7a56c0fd0c43d4803634f30479187a, user_id=9ace48889c484e4fb5fed0f8f80c3288, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:02.020462 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: WARNING py.warnings [None req-9f5fbaa0-e598-46d8-a7a9-a2cc4ccd8f04 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 10:26:02.020462 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: warnings.warn(msg) Aug 05 10:26:02.020462 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: Aug 05 10:26:02.022398 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5fbaa0-e598-46d8-a7a9-a2cc4ccd8f04 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:02.024659 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe953e-8cd8-4d83-a23d-f7498ceea27c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:02.029668 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be2a7b-028f-4349-9857-e61f336bcfe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:02.030314 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be2a7b-028f-4349-9857-e61f336bcfe9 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:02.030596 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be2a7b-028f-4349-9857-e61f336bcfe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:02.030872 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be2a7b-028f-4349-9857-e61f336bcfe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:02.031582 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4be2a7b-028f-4349-9857-e61f336bcfe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:02.033319 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4be2a7b-028f-4349-9857-e61f336bcfe9 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:02.038185 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 782/1571] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:26:01 2022] PUT /identity/v3/projects/5a7a56c0fd0c43d4803634f30479187a/users/9ace48889c484e4fb5fed0f8f80c3288/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:02.039274 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5fbaa0-e598-46d8-a7a9-a2cc4ccd8f04 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:02.045217 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 790/1572] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:02.045997 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be2a7b-028f-4349-9857-e61f336bcfe9 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:02.056006 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be2a7b-028f-4349-9857-e61f336bcfe9 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:02.109375 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5fbaa0-e598-46d8-a7a9-a2cc4ccd8f04 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:02.113776 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab8c47-c3e0-4e7b-8156-ab592dcaa4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:02.114352 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e8b3f-a8fa-496f-9158-75782d58a5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:02.114540 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab8c47-c3e0-4e7b-8156-ab592dcaa4b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:02.114649 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab8c47-c3e0-4e7b-8156-ab592dcaa4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:02.114918 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab8c47-c3e0-4e7b-8156-ab592dcaa4b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:02.115037 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e8b3f-a8fa-496f-9158-75782d58a5c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:02.115287 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e8b3f-a8fa-496f-9158-75782d58a5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:02.115565 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e8b3f-a8fa-496f-9158-75782d58a5c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:02.115661 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daab8c47-c3e0-4e7b-8156-ab592dcaa4b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:02.116231 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329e8b3f-a8fa-496f-9158-75782d58a5c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:02.117698 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daab8c47-c3e0-4e7b-8156-ab592dcaa4b3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:02.118133 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329e8b3f-a8fa-496f-9158-75782d58a5c8 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:02.146589 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 783/1573] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:26:02 2022] POST /identity/v3/projects => generated 377 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:26:02.150632 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 791/1574] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:26:02 2022] POST /identity/v3/projects => generated 387 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:26:02.150632 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab8c47-c3e0-4e7b-8156-ab592dcaa4b3 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:02.151232 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e8b3f-a8fa-496f-9158-75782d58a5c8 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:02.213439 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab8c47-c3e0-4e7b-8156-ab592dcaa4b3 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:02.218073 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b3f215-cb95-462c-be69-4042a1864982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:02.218602 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3f215-cb95-462c-be69-4042a1864982 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:02.218878 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3f215-cb95-462c-be69-4042a1864982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:02.219180 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3f215-cb95-462c-be69-4042a1864982 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:02.219841 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b3f215-cb95-462c-be69-4042a1864982 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:02.222072 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b3f215-cb95-462c-be69-4042a1864982 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:02.223610 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e8b3f-a8fa-496f-9158-75782d58a5c8 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:02.229031 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660023e3-399e-4b4c-bc35-763705a70274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:02.229694 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660023e3-399e-4b4c-bc35-763705a70274 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:02.229976 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660023e3-399e-4b4c-bc35-763705a70274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:02.230246 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660023e3-399e-4b4c-bc35-763705a70274 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:02.230943 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660023e3-399e-4b4c-bc35-763705a70274 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:02.232854 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660023e3-399e-4b4c-bc35-763705a70274 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:02.244222 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:02.253036 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 792/1575] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:26:02 2022] POST /identity/v3/projects => generated 367 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:26:02.253826 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b3f215-cb95-462c-be69-4042a1864982 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:02.266951 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 784/1576] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:26:02 2022] POST /identity/v3/projects => generated 367 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:26:02.266951 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660023e3-399e-4b4c-bc35-763705a70274 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:02.270325 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:02.321895 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b3f215-cb95-462c-be69-4042a1864982 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:02.325991 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660023e3-399e-4b4c-bc35-763705a70274 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:02.326480 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f07059-aa27-4295-b483-c5532c50eac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:02.327132 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f07059-aa27-4295-b483-c5532c50eac8 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:02.327412 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f07059-aa27-4295-b483-c5532c50eac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:02.327684 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f07059-aa27-4295-b483-c5532c50eac8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:02.328349 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f07059-aa27-4295-b483-c5532c50eac8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:02.330317 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f07059-aa27-4295-b483-c5532c50eac8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:02.331430 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8823e745-1be8-40dd-adea-94947aab9f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:02.332062 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8823e745-1be8-40dd-adea-94947aab9f31 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:02.332348 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8823e745-1be8-40dd-adea-94947aab9f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:02.332621 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8823e745-1be8-40dd-adea-94947aab9f31 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:02.333320 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8823e745-1be8-40dd-adea-94947aab9f31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:02.335305 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8823e745-1be8-40dd-adea-94947aab9f31 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:02.360420 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 793/1577] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:26:02 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:26:02.362877 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f3d88-704d-40ae-aa8d-2604a09ced28 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:02.363199 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f3d88-704d-40ae-aa8d-2604a09ced28 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:02.363455 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f3d88-704d-40ae-aa8d-2604a09ced28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:02.365445 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 785/1578] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:26:02 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:26:02.366199 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8823e745-1be8-40dd-adea-94947aab9f31 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:02.383239 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-0c2f3d88-704d-40ae-aa8d-2604a09ced28 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:02.384439 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0c2f3d88-704d-40ae-aa8d-2604a09ced28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:02.430175 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8823e745-1be8-40dd-adea-94947aab9f31 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:02.435245 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25810726-d9ef-4593-990c-b699d482c25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:02.435916 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25810726-d9ef-4593-990c-b699d482c25e admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:02.436203 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25810726-d9ef-4593-990c-b699d482c25e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:02.436474 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25810726-d9ef-4593-990c-b699d482c25e admin admin] PATH_INFO: `/v3/projects/9ea83951557a48fcaab832aea2273453/users/a3b4d90bcfc34d2d9ba1c40316e09858/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:02.445185 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 794/1579] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:02.446189 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f3d88-704d-40ae-aa8d-2604a09ced28 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:02.451838 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25810726-d9ef-4593-990c-b699d482c25e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ea83951557a48fcaab832aea2273453, user_id=a3b4d90bcfc34d2d9ba1c40316e09858, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:02.454524 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25810726-d9ef-4593-990c-b699d482c25e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:02.458326 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f3d88-704d-40ae-aa8d-2604a09ced28 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:02.463302 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d12fe5-2378-4c65-bc89-56cbca804584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:02.464042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d12fe5-2378-4c65-bc89-56cbca804584 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:02.464448 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d12fe5-2378-4c65-bc89-56cbca804584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:02.465083 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d12fe5-2378-4c65-bc89-56cbca804584 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:02.465559 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d12fe5-2378-4c65-bc89-56cbca804584 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:02.467543 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d12fe5-2378-4c65-bc89-56cbca804584 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:02.469301 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 786/1580] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:26:02 2022] PUT /identity/v3/projects/9ea83951557a48fcaab832aea2273453/users/a3b4d90bcfc34d2d9ba1c40316e09858/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:02.473245 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25810726-d9ef-4593-990c-b699d482c25e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:02.494713 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 795/1581] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:26:02 2022] POST /identity/v3/users => generated 289 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:26:02.494713 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d12fe5-2378-4c65-bc89-56cbca804584 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:02.504242 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d12fe5-2378-4c65-bc89-56cbca804584 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:02.531679 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25810726-d9ef-4593-990c-b699d482c25e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:02.536324 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639cb41-7fa8-4ff0-ae82-2bc23ef0a895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:02.537283 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639cb41-7fa8-4ff0-ae82-2bc23ef0a895 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:02.537559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639cb41-7fa8-4ff0-ae82-2bc23ef0a895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:02.537830 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639cb41-7fa8-4ff0-ae82-2bc23ef0a895 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:02.538489 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5639cb41-7fa8-4ff0-ae82-2bc23ef0a895 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:02.540445 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5639cb41-7fa8-4ff0-ae82-2bc23ef0a895 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:02.562887 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab022202-a16b-4f6d-aa4b-c4fafd9dfe31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:02.563465 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 787/1582] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:26:02 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:26:02.563586 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab022202-a16b-4f6d-aa4b-c4fafd9dfe31 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:02.563812 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab022202-a16b-4f6d-aa4b-c4fafd9dfe31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:02.564084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab022202-a16b-4f6d-aa4b-c4fafd9dfe31 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:02.564267 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639cb41-7fa8-4ff0-ae82-2bc23ef0a895 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:02.565526 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab022202-a16b-4f6d-aa4b-c4fafd9dfe31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:02.567908 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab022202-a16b-4f6d-aa4b-c4fafd9dfe31 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:02.569683 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:02.575525 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639cb41-7fa8-4ff0-ae82-2bc23ef0a895 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:02.579335 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:02.582669 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc961e-d2b7-4972-8dcd-a7b561ac5262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:02.583397 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc961e-d2b7-4972-8dcd-a7b561ac5262 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:02.583710 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc961e-d2b7-4972-8dcd-a7b561ac5262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:02.583957 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc961e-d2b7-4972-8dcd-a7b561ac5262 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:02.584735 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfc961e-d2b7-4972-8dcd-a7b561ac5262 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:02.586799 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfc961e-d2b7-4972-8dcd-a7b561ac5262 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:02.592262 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 796/1583] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:26:02 2022] POST /identity/v3/users => generated 284 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:26:02.593287 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab022202-a16b-4f6d-aa4b-c4fafd9dfe31 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:02.604634 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab022202-a16b-4f6d-aa4b-c4fafd9dfe31 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:02.609238 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ff9e42-46e5-4db5-aeb5-bff0762e921c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:02.609841 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff9e42-46e5-4db5-aeb5-bff0762e921c admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:02.610118 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff9e42-46e5-4db5-aeb5-bff0762e921c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:02.610386 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff9e42-46e5-4db5-aeb5-bff0762e921c admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:02.613943 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 788/1584] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:26:02 2022] POST /identity/v3/users => generated 284 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:26:02.614102 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ff9e42-46e5-4db5-aeb5-bff0762e921c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:02.614215 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc961e-d2b7-4972-8dcd-a7b561ac5262 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:02.614312 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ff9e42-46e5-4db5-aeb5-bff0762e921c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:02.622247 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc961e-d2b7-4972-8dcd-a7b561ac5262 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:02.628359 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e72be7-7801-4809-8ed2-0a2f6a7ec1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:02.628967 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e72be7-7801-4809-8ed2-0a2f6a7ec1d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:02.629243 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e72be7-7801-4809-8ed2-0a2f6a7ec1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:02.629512 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e72be7-7801-4809-8ed2-0a2f6a7ec1d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:02.630193 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e72be7-7801-4809-8ed2-0a2f6a7ec1d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:02.632190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e72be7-7801-4809-8ed2-0a2f6a7ec1d1 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:02.636051 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 797/1585] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:26:02 2022] POST /identity/v3/users => generated 292 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:26:02.639914 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ff9e42-46e5-4db5-aeb5-bff0762e921c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:02.641526 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c7ff9e42-46e5-4db5-aeb5-bff0762e921c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:02.659271 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 789/1586] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:26:02 2022] POST /identity/v3/users => generated 287 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:26:02.659271 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5729a0-e155-4d45-9c47-e00b529051e7 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:02.659271 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5729a0-e155-4d45-9c47-e00b529051e7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:02.659271 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5729a0-e155-4d45-9c47-e00b529051e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:02.676814 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-2b5729a0-e155-4d45-9c47-e00b529051e7 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:02.678012 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2b5729a0-e155-4d45-9c47-e00b529051e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:02.685766 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 790/1587] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:02.686462 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5729a0-e155-4d45-9c47-e00b529051e7 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:02.695842 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5729a0-e155-4d45-9c47-e00b529051e7 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:02.700698 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca821e-98cb-45ac-aeda-159f83b7807f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:02.701315 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca821e-98cb-45ac-aeda-159f83b7807f admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:02.701586 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca821e-98cb-45ac-aeda-159f83b7807f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:02.701853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca821e-98cb-45ac-aeda-159f83b7807f admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:02.702547 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ca821e-98cb-45ac-aeda-159f83b7807f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:02.704208 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ca821e-98cb-45ac-aeda-159f83b7807f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:02.709282 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ff9e42-46e5-4db5-aeb5-bff0762e921c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:02.712928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e117f-814e-48e1-b041-4f2a32c503f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:02.713554 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e117f-814e-48e1-b041-4f2a32c503f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:02.713832 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e117f-814e-48e1-b041-4f2a32c503f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:02.714106 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e117f-814e-48e1-b041-4f2a32c503f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:02.718064 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 791/1588] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:02.718064 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca821e-98cb-45ac-aeda-159f83b7807f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:02.723862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4e117f-814e-48e1-b041-4f2a32c503f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:02.725312 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4e117f-814e-48e1-b041-4f2a32c503f3 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:02.725538 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca821e-98cb-45ac-aeda-159f83b7807f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:02.730002 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dfcd1d-fd1d-4685-8a21-97a1f46294a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:02.730591 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfcd1d-fd1d-4685-8a21-97a1f46294a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:02.730868 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfcd1d-fd1d-4685-8a21-97a1f46294a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:02.731165 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfcd1d-fd1d-4685-8a21-97a1f46294a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:02.731842 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dfcd1d-fd1d-4685-8a21-97a1f46294a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:02.733413 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dfcd1d-fd1d-4685-8a21-97a1f46294a3 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:02.735313 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 798/1589] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:26:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:26:02.736163 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e117f-814e-48e1-b041-4f2a32c503f3 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:02.743898 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 792/1590] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:02.746200 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e117f-814e-48e1-b041-4f2a32c503f3 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:02.747504 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dfcd1d-fd1d-4685-8a21-97a1f46294a3 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:02.751185 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b7f8f-5142-4457-90c1-11584c843e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:02.751938 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b7f8f-5142-4457-90c1-11584c843e06 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:02.752301 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b7f8f-5142-4457-90c1-11584c843e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:02.753237 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b7f8f-5142-4457-90c1-11584c843e06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:02.754092 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12b7f8f-5142-4457-90c1-11584c843e06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:02.755638 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12b7f8f-5142-4457-90c1-11584c843e06 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:02.757712 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dfcd1d-fd1d-4685-8a21-97a1f46294a3 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:02.762565 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c8468-b873-4d8a-8955-b510c29b8262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:02.763217 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c8468-b873-4d8a-8955-b510c29b8262 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:02.763492 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c8468-b873-4d8a-8955-b510c29b8262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:02.763762 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c8468-b873-4d8a-8955-b510c29b8262 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:02.764451 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3c8468-b873-4d8a-8955-b510c29b8262 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:02.766042 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3c8468-b873-4d8a-8955-b510c29b8262 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:02.766230 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 799/1591] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:02.769759 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b7f8f-5142-4457-90c1-11584c843e06 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:02.780528 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 793/1592] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:02.780877 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b7f8f-5142-4457-90c1-11584c843e06 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:02.780986 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c8468-b873-4d8a-8955-b510c29b8262 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:02.785093 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51d208-3a0a-4f53-99a1-bd5dc18813c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:02.785903 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51d208-3a0a-4f53-99a1-bd5dc18813c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:02.786185 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51d208-3a0a-4f53-99a1-bd5dc18813c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:02.786461 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51d208-3a0a-4f53-99a1-bd5dc18813c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:02.788075 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c51d208-3a0a-4f53-99a1-bd5dc18813c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:02.789705 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c51d208-3a0a-4f53-99a1-bd5dc18813c5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:02.793079 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c8468-b873-4d8a-8955-b510c29b8262 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:02.800764 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8041d0b8-d87a-4fc7-ae4a-f5af809c285c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:02.801613 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 800/1593] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:02.801746 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041d0b8-d87a-4fc7-ae4a-f5af809c285c admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:02.802137 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041d0b8-d87a-4fc7-ae4a-f5af809c285c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:02.802618 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041d0b8-d87a-4fc7-ae4a-f5af809c285c admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:02.802731 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51d208-3a0a-4f53-99a1-bd5dc18813c5 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:02.803641 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8041d0b8-d87a-4fc7-ae4a-f5af809c285c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:02.806528 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8041d0b8-d87a-4fc7-ae4a-f5af809c285c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:02.814408 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51d208-3a0a-4f53-99a1-bd5dc18813c5 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:02.818929 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea29d21-26fb-4e82-a118-c5e766596794 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:02.819884 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea29d21-26fb-4e82-a118-c5e766596794 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:02.819884 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea29d21-26fb-4e82-a118-c5e766596794 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:02.820116 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea29d21-26fb-4e82-a118-c5e766596794 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:02.825079 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 794/1594] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:02.825079 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8041d0b8-d87a-4fc7-ae4a-f5af809c285c admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:02.831592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea29d21-26fb-4e82-a118-c5e766596794 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:02.833061 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea29d21-26fb-4e82-a118-c5e766596794 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:02.835406 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8041d0b8-d87a-4fc7-ae4a-f5af809c285c admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:02.842158 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f91c68c-c977-47ad-9bb7-247bc43a3d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:02.843136 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91c68c-c977-47ad-9bb7-247bc43a3d39 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:02.843571 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91c68c-c977-47ad-9bb7-247bc43a3d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:02.844058 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 801/1595] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:26:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:26:02.844428 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91c68c-c977-47ad-9bb7-247bc43a3d39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:02.844718 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea29d21-26fb-4e82-a118-c5e766596794 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:02.845469 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f91c68c-c977-47ad-9bb7-247bc43a3d39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:02.847484 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f91c68c-c977-47ad-9bb7-247bc43a3d39 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:02.854580 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea29d21-26fb-4e82-a118-c5e766596794 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:02.859258 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd119e9-dfc5-45c9-a9bd-3a9bb0ed8caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:02.859977 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd119e9-dfc5-45c9-a9bd-3a9bb0ed8caf admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:02.860397 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 795/1596] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:02.861850 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f91c68c-c977-47ad-9bb7-247bc43a3d39 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:02.863438 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd119e9-dfc5-45c9-a9bd-3a9bb0ed8caf admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:02.863740 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd119e9-dfc5-45c9-a9bd-3a9bb0ed8caf admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:02.864466 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd119e9-dfc5-45c9-a9bd-3a9bb0ed8caf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:02.866111 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd119e9-dfc5-45c9-a9bd-3a9bb0ed8caf admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:02.873534 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f91c68c-c977-47ad-9bb7-247bc43a3d39 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:02.879119 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f7fc2-d41f-4945-aae8-e51519716635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:02.879847 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f7fc2-d41f-4945-aae8-e51519716635 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:02.883533 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f7fc2-d41f-4945-aae8-e51519716635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:02.883639 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 802/1597] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:02.883735 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f7fc2-d41f-4945-aae8-e51519716635 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:02.883735 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f7fc2-d41f-4945-aae8-e51519716635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:02.883905 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd119e9-dfc5-45c9-a9bd-3a9bb0ed8caf admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:02.884089 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f7fc2-d41f-4945-aae8-e51519716635 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:02.895091 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd119e9-dfc5-45c9-a9bd-3a9bb0ed8caf admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:02.895279 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 796/1598] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:02.896169 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f7fc2-d41f-4945-aae8-e51519716635 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:02.903459 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47450daf-c68a-4951-92a7-b7873d0e6c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:02.905157 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47450daf-c68a-4951-92a7-b7873d0e6c12 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:02.905443 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47450daf-c68a-4951-92a7-b7873d0e6c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:02.905712 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47450daf-c68a-4951-92a7-b7873d0e6c12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:02.906538 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47450daf-c68a-4951-92a7-b7873d0e6c12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:02.908502 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47450daf-c68a-4951-92a7-b7873d0e6c12 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:02.909153 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f7fc2-d41f-4945-aae8-e51519716635 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:02.915791 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ed888e-70ac-476f-93c7-c862dac9dc6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:02.916422 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed888e-70ac-476f-93c7-c862dac9dc6e admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:02.916827 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed888e-70ac-476f-93c7-c862dac9dc6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:02.917038 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed888e-70ac-476f-93c7-c862dac9dc6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:02.917738 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ed888e-70ac-476f-93c7-c862dac9dc6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:02.919841 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ed888e-70ac-476f-93c7-c862dac9dc6e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:02.920024 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 803/1599] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:02.920571 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47450daf-c68a-4951-92a7-b7873d0e6c12 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:02.933288 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47450daf-c68a-4951-92a7-b7873d0e6c12 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:02.936511 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 797/1600] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:02.937281 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ed888e-70ac-476f-93c7-c862dac9dc6e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:02.942432 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc83a206-42cd-432e-89b3-9fb86f213620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:02.943168 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc83a206-42cd-432e-89b3-9fb86f213620 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:02.943520 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc83a206-42cd-432e-89b3-9fb86f213620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:02.943844 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc83a206-42cd-432e-89b3-9fb86f213620 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:02.944589 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc83a206-42cd-432e-89b3-9fb86f213620 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:02.946451 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc83a206-42cd-432e-89b3-9fb86f213620 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:02.950056 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ed888e-70ac-476f-93c7-c862dac9dc6e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:02.954952 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba3723-6b2f-4546-9495-86ecf2299251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:02.955724 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba3723-6b2f-4546-9495-86ecf2299251 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:02.956256 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba3723-6b2f-4546-9495-86ecf2299251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:02.956606 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba3723-6b2f-4546-9495-86ecf2299251 admin admin] PATH_INFO: `/v3/projects/40807f4af05048deae68cf3515433261/users/7e7eb355fac04252a4be1f8e60037924/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:02.958193 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 804/1601] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:02.961684 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc83a206-42cd-432e-89b3-9fb86f213620 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:02.972269 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc83a206-42cd-432e-89b3-9fb86f213620 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:02.975967 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ba3723-6b2f-4546-9495-86ecf2299251 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40807f4af05048deae68cf3515433261, user_id=7e7eb355fac04252a4be1f8e60037924, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:02.977786 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb734ab5-d32f-4214-90a3-719f46dc6b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:02.978669 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb734ab5-d32f-4214-90a3-719f46dc6b13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:02.978877 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ba3723-6b2f-4546-9495-86ecf2299251 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:02.979015 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb734ab5-d32f-4214-90a3-719f46dc6b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:02.979259 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb734ab5-d32f-4214-90a3-719f46dc6b13 admin admin] PATH_INFO: `/v3/projects/36a1fa6b4a754e47afb99e9b237ea400/users/97f671de92d240ae85ca4893d1ff04e5/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:02.997573 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb734ab5-d32f-4214-90a3-719f46dc6b13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36a1fa6b4a754e47afb99e9b237ea400, user_id=97f671de92d240ae85ca4893d1ff04e5, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:02.998938 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 798/1602] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:26:02 2022] PUT /identity/v3/projects/40807f4af05048deae68cf3515433261/users/7e7eb355fac04252a4be1f8e60037924/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:03.000773 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb734ab5-d32f-4214-90a3-719f46dc6b13 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:03.003839 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba3723-6b2f-4546-9495-86ecf2299251 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:03.022808 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 805/1603] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:26:02 2022] PUT /identity/v3/projects/36a1fa6b4a754e47afb99e9b237ea400/users/97f671de92d240ae85ca4893d1ff04e5/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:03.024312 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb734ab5-d32f-4214-90a3-719f46dc6b13 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:03.078165 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba3723-6b2f-4546-9495-86ecf2299251 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:03.084156 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:03.108579 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb734ab5-d32f-4214-90a3-719f46dc6b13 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:03.108799 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:03.115601 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9769cf8-77bb-444e-893a-9d8454225b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:03.116653 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9769cf8-77bb-444e-893a-9d8454225b15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:03.116986 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9769cf8-77bb-444e-893a-9d8454225b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:03.117362 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9769cf8-77bb-444e-893a-9d8454225b15 admin admin] PATH_INFO: `/v3/projects/a33f145210734116a61371c8e6ba404a/users/20eb8a3abe034044a1785f6b7f6a5d4b/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:03.136356 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9769cf8-77bb-444e-893a-9d8454225b15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a33f145210734116a61371c8e6ba404a, user_id=20eb8a3abe034044a1785f6b7f6a5d4b, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:03.148816 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9769cf8-77bb-444e-893a-9d8454225b15 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:03.163533 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b868432f-c811-4d7a-929a-7f032ab97239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:03.163959 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b868432f-c811-4d7a-929a-7f032ab97239 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:03.164170 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b868432f-c811-4d7a-929a-7f032ab97239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:03.164416 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b868432f-c811-4d7a-929a-7f032ab97239 admin admin] PATH_INFO: `/v3/projects/59b286e5a3bc416f996a249eadb38c11/users/805164588f414dd084eae660fd52eda3/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:03.172811 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 806/1604] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:26:03 2022] PUT /identity/v3/projects/a33f145210734116a61371c8e6ba404a/users/20eb8a3abe034044a1785f6b7f6a5d4b/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:03.173636 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9769cf8-77bb-444e-893a-9d8454225b15 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:03.187946 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b868432f-c811-4d7a-929a-7f032ab97239 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59b286e5a3bc416f996a249eadb38c11, user_id=805164588f414dd084eae660fd52eda3, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:03.192999 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b868432f-c811-4d7a-929a-7f032ab97239 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:03.215064 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 799/1605] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:26:02 2022] PUT /identity/v3/projects/59b286e5a3bc416f996a249eadb38c11/users/805164588f414dd084eae660fd52eda3/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:03.216913 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b868432f-c811-4d7a-929a-7f032ab97239 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:03.273033 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9769cf8-77bb-444e-893a-9d8454225b15 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:03.355833 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d6b0d-294f-46f4-96e9-1bfdd8c2a2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:03.357456 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d6b0d-294f-46f4-96e9-1bfdd8c2a2cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:03.357957 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d6b0d-294f-46f4-96e9-1bfdd8c2a2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:03.358398 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d6b0d-294f-46f4-96e9-1bfdd8c2a2cb admin admin] PATH_INFO: `/v3/projects/588c79a0604e49f49123980f0e9af63a/users/706dbfa719c04fd0a3704ebd7eb812cd/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:03.366153 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b868432f-c811-4d7a-929a-7f032ab97239 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:03.381226 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bf6d9e-3aea-40b4-b613-e760398e181e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:03.382102 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf6d9e-3aea-40b4-b613-e760398e181e admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:03.382534 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf6d9e-3aea-40b4-b613-e760398e181e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:03.382846 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf6d9e-3aea-40b4-b613-e760398e181e admin admin] PATH_INFO: `/v3/projects/78c1e92af310437db4b79bfed67fa76a/users/3478f4fce4034422b8313310a834b364/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:03.386048 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d6b0d-294f-46f4-96e9-1bfdd8c2a2cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=588c79a0604e49f49123980f0e9af63a, user_id=706dbfa719c04fd0a3704ebd7eb812cd, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:03.390329 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d6b0d-294f-46f4-96e9-1bfdd8c2a2cb admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:03.410029 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bf6d9e-3aea-40b4-b613-e760398e181e admin admin] RBAC: Authorizing `identity:create_grant(project_id=78c1e92af310437db4b79bfed67fa76a, user_id=3478f4fce4034422b8313310a834b364, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:03.415629 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bf6d9e-3aea-40b4-b613-e760398e181e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:03.418423 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 807/1606] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:26:03 2022] PUT /identity/v3/projects/588c79a0604e49f49123980f0e9af63a/users/706dbfa719c04fd0a3704ebd7eb812cd/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:03.431270 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7132e637-301a-4982-9a90-dfa7eed95480 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:03.431817 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7132e637-301a-4982-9a90-dfa7eed95480 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:03.432145 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7132e637-301a-4982-9a90-dfa7eed95480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:03.434825 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 800/1607] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:26:03 2022] PUT /identity/v3/projects/78c1e92af310437db4b79bfed67fa76a/users/3478f4fce4034422b8313310a834b364/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:03.436189 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e203fa-766e-4e7c-a490-379e6af71d56 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:03.436571 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e203fa-766e-4e7c-a490-379e6af71d56 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:03.436961 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e203fa-766e-4e7c-a490-379e6af71d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:03.470261 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-7132e637-301a-4982-9a90-dfa7eed95480 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:03.471652 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7132e637-301a-4982-9a90-dfa7eed95480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:03.475681 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-17e203fa-766e-4e7c-a490-379e6af71d56 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:03.477258 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-17e203fa-766e-4e7c-a490-379e6af71d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:03.590665 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 808/1608] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:03.592189 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c347d9b-3e3c-464f-a064-8633ec49eb94 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:03.592694 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c347d9b-3e3c-464f-a064-8633ec49eb94 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:03.592997 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c347d9b-3e3c-464f-a064-8633ec49eb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:03.600683 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 801/1609] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:03.601960 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:03.603101 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad41447-1dd2-4fba-b7a5-43563e93483a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:03.603628 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad41447-1dd2-4fba-b7a5-43563e93483a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:03.603893 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad41447-1dd2-4fba-b7a5-43563e93483a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:03.628907 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1c347d9b-3e3c-464f-a064-8633ec49eb94 None None] MFA Rules not processed for user `20eb8a3abe034044a1785f6b7f6a5d4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:03.631147 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1c347d9b-3e3c-464f-a064-8633ec49eb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:03.638859 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-5ad41447-1dd2-4fba-b7a5-43563e93483a None None] MFA Rules not processed for user `805164588f414dd084eae660fd52eda3`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:03.641039 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5ad41447-1dd2-4fba-b7a5-43563e93483a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:03.683090 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:03.749319 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 802/1610] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:03.750315 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 809/1611] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:03.750954 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9abc6-808c-40d6-b169-7e8fa9d53ce4 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:03.751315 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9abc6-808c-40d6-b169-7e8fa9d53ce4 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:03.751805 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9abc6-808c-40d6-b169-7e8fa9d53ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:03.752154 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b4b73-276f-4b2a-962b-ecbad3bead88 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:03.752609 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b4b73-276f-4b2a-962b-ecbad3bead88 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:03.753026 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b4b73-276f-4b2a-962b-ecbad3bead88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:03.776097 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c3a9abc6-808c-40d6-b169-7e8fa9d53ce4 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:03.777283 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c3a9abc6-808c-40d6-b169-7e8fa9d53ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:03.789288 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ee9b4b73-276f-4b2a-962b-ecbad3bead88 None None] MFA Rules not processed for user `3478f4fce4034422b8313310a834b364`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:03.791302 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ee9b4b73-276f-4b2a-962b-ecbad3bead88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:03.794688 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 803/1612] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:03.795709 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9abc6-808c-40d6-b169-7e8fa9d53ce4 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:03.872372 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 810/1613] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:03.873111 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9b4b73-276f-4b2a-962b-ecbad3bead88 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:03.874608 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9abc6-808c-40d6-b169-7e8fa9d53ce4 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:03.883863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9b4b73-276f-4b2a-962b-ecbad3bead88 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:03.887923 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65fe38e-315a-4aaa-99e7-a16a9af7b226 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:03.888768 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc0b90-9095-4726-921b-7e0e5b43219b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:03.889674 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65fe38e-315a-4aaa-99e7-a16a9af7b226 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:03.889819 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc0b90-9095-4726-921b-7e0e5b43219b service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:03.889930 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65fe38e-315a-4aaa-99e7-a16a9af7b226 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:03.889930 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65fe38e-315a-4aaa-99e7-a16a9af7b226 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:03.890226 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc0b90-9095-4726-921b-7e0e5b43219b service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:03.890708 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc0b90-9095-4726-921b-7e0e5b43219b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:03.907845 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65fe38e-315a-4aaa-99e7-a16a9af7b226 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:03.910514 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65fe38e-315a-4aaa-99e7-a16a9af7b226 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:03.920953 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddc0b90-9095-4726-921b-7e0e5b43219b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:03.924190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddc0b90-9095-4726-921b-7e0e5b43219b service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:03.939101 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 811/1614] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:26:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:26:03.940456 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65fe38e-315a-4aaa-99e7-a16a9af7b226 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:03.952189 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65fe38e-315a-4aaa-99e7-a16a9af7b226 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:03.961036 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7afc28-1087-4115-b38e-f74dd6460009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:03.962280 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7afc28-1087-4115-b38e-f74dd6460009 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:03.962570 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 804/1615] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:26:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:26:03.963337 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc0b90-9095-4726-921b-7e0e5b43219b service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:03.964001 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7afc28-1087-4115-b38e-f74dd6460009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:03.964572 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7afc28-1087-4115-b38e-f74dd6460009 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:03.965843 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7afc28-1087-4115-b38e-f74dd6460009 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:03.968938 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7afc28-1087-4115-b38e-f74dd6460009 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:04.023166 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 812/1616] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:26:03 2022] POST /identity/v3/projects => generated 367 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:26:04.024275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7afc28-1087-4115-b38e-f74dd6460009 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:04.110474 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc0b90-9095-4726-921b-7e0e5b43219b service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:04.156650 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:04.175009 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7afc28-1087-4115-b38e-f74dd6460009 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:04.182222 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87861f2-ffb2-44f4-bfc1-3dde6778a50f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:04.183780 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87861f2-ffb2-44f4-bfc1-3dde6778a50f service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:04.184638 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87861f2-ffb2-44f4-bfc1-3dde6778a50f service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:04.186283 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87861f2-ffb2-44f4-bfc1-3dde6778a50f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:04.236375 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ba8af-5651-4296-8014-8fde8c6cadc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:04.237769 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ba8af-5651-4296-8014-8fde8c6cadc8 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:04.240414 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ba8af-5651-4296-8014-8fde8c6cadc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:04.240809 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ba8af-5651-4296-8014-8fde8c6cadc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:04.323881 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87861f2-ffb2-44f4-bfc1-3dde6778a50f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:04.325654 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87861f2-ffb2-44f4-bfc1-3dde6778a50f service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:04.337801 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:04.339669 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 813/1617] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:26:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:26:04.341178 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675f8ed-2faa-4b66-a493-eae0f66ad531 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:04.341660 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675f8ed-2faa-4b66-a493-eae0f66ad531 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:04.342268 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675f8ed-2faa-4b66-a493-eae0f66ad531 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:04.343813 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 814/1618] 10.222.0.40 () {54 vars in 962 bytes} [Fri Aug 5 10:26:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:26:04.350132 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1675f8ed-2faa-4b66-a493-eae0f66ad531 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:04.366094 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1675f8ed-2faa-4b66-a493-eae0f66ad531 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:04.371716 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbbdc4b-75c7-4425-b36f-635136fbdb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:04.372592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbbdc4b-75c7-4425-b36f-635136fbdb10 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:04.373000 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbbdc4b-75c7-4425-b36f-635136fbdb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:04.373397 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbbdc4b-75c7-4425-b36f-635136fbdb10 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:04.374170 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbbdc4b-75c7-4425-b36f-635136fbdb10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:04.376401 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbbdc4b-75c7-4425-b36f-635136fbdb10 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:04.411651 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8ba8af-5651-4296-8014-8fde8c6cadc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:04.415286 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8ba8af-5651-4296-8014-8fde8c6cadc8 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:04.418391 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 815/1619] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:26:04 2022] POST /identity/v3/users => generated 290 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:26:04.423769 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c18c0e-ede0-4bcb-8219-9a1699e48975 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:04.424273 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c18c0e-ede0-4bcb-8219-9a1699e48975 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:04.424663 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c18c0e-ede0-4bcb-8219-9a1699e48975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:04.439399 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 805/1620] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:26:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:26:04.441635 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ba8af-5651-4296-8014-8fde8c6cadc8 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:04.469480 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ba8af-5651-4296-8014-8fde8c6cadc8 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:04.482229 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19719468-9b78-4e9e-9b23-5b22de8919cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:04.484748 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19719468-9b78-4e9e-9b23-5b22de8919cb admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:04.488207 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19719468-9b78-4e9e-9b23-5b22de8919cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:04.488659 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19719468-9b78-4e9e-9b23-5b22de8919cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:04.489743 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19719468-9b78-4e9e-9b23-5b22de8919cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:04.493191 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19719468-9b78-4e9e-9b23-5b22de8919cb admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:04.504926 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-72c18c0e-ede0-4bcb-8219-9a1699e48975 None None] MFA Rules not processed for user `b938f1f41db94b0eafe95bc8eeaae175`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:04.506099 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-72c18c0e-ede0-4bcb-8219-9a1699e48975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:04.518256 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 806/1621] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:04.519441 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19719468-9b78-4e9e-9b23-5b22de8919cb admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:04.542370 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19719468-9b78-4e9e-9b23-5b22de8919cb admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:04.555791 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bfbfde-693a-496c-8656-4c3b2fc757dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:04.556665 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bfbfde-693a-496c-8656-4c3b2fc757dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:04.557051 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bfbfde-693a-496c-8656-4c3b2fc757dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:04.557426 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bfbfde-693a-496c-8656-4c3b2fc757dd admin admin] PATH_INFO: `/v3/projects/d806e1db3b3b4db0adc10ded22256192/users/28c01479ae554885916fd5ab402ab897/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:04.592654 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bfbfde-693a-496c-8656-4c3b2fc757dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d806e1db3b3b4db0adc10ded22256192, user_id=28c01479ae554885916fd5ab402ab897, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:04.595822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bfbfde-693a-496c-8656-4c3b2fc757dd admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:04.600941 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 816/1622] 10.222.0.40 () {58 vars in 1059 bytes} [Fri Aug 5 10:26:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:04.612068 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c18c0e-ede0-4bcb-8219-9a1699e48975 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:04.625152 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 807/1623] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:26:04 2022] PUT /identity/v3/projects/d806e1db3b3b4db0adc10ded22256192/users/28c01479ae554885916fd5ab402ab897/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:04.629052 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bfbfde-693a-496c-8656-4c3b2fc757dd admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:04.638384 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c18c0e-ede0-4bcb-8219-9a1699e48975 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:04.764272 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bfbfde-693a-496c-8656-4c3b2fc757dd admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:04.775647 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e969bc47-1605-48ce-9271-70860d3a1727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:04.776297 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e969bc47-1605-48ce-9271-70860d3a1727 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:04.776583 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e969bc47-1605-48ce-9271-70860d3a1727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:04.776858 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e969bc47-1605-48ce-9271-70860d3a1727 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:04.777566 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e969bc47-1605-48ce-9271-70860d3a1727 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:04.779601 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e969bc47-1605-48ce-9271-70860d3a1727 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:04.781564 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:04.801537 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 808/1624] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 10:26:04.809177 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e969bc47-1605-48ce-9271-70860d3a1727 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:04.818469 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc1d010-e625-4277-89fc-683faafc6f7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:04.819227 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc1d010-e625-4277-89fc-683faafc6f7d service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:04.819511 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc1d010-e625-4277-89fc-683faafc6f7d service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:04.821939 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc1d010-e625-4277-89fc-683faafc6f7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:04.830396 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e969bc47-1605-48ce-9271-70860d3a1727 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:04.845154 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e62a4-8f8c-4b51-93c8-10ac209dfffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:04.846018 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e62a4-8f8c-4b51-93c8-10ac209dfffd admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:04.849983 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e62a4-8f8c-4b51-93c8-10ac209dfffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:04.850316 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e62a4-8f8c-4b51-93c8-10ac209dfffd admin admin] PATH_INFO: `/v3/projects/d806e1db3b3b4db0adc10ded22256192/users/28c01479ae554885916fd5ab402ab897/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:04.858975 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770e62a4-8f8c-4b51-93c8-10ac209dfffd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d806e1db3b3b4db0adc10ded22256192, user_id=28c01479ae554885916fd5ab402ab897, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:04.871153 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770e62a4-8f8c-4b51-93c8-10ac209dfffd admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:04.898637 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 809/1625] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:26:04 2022] PUT /identity/v3/projects/d806e1db3b3b4db0adc10ded22256192/users/28c01479ae554885916fd5ab402ab897/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:04.910091 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa00fa-2d7d-492f-b57b-ad29fd427377 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:04.910331 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa00fa-2d7d-492f-b57b-ad29fd427377 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:04.910725 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa00fa-2d7d-492f-b57b-ad29fd427377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:04.965557 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc1d010-e625-4277-89fc-683faafc6f7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:04.968695 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc1d010-e625-4277-89fc-683faafc6f7d service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:04.980516 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-8efa00fa-2d7d-492f-b57b-ad29fd427377 None None] MFA Rules not processed for user `28c01479ae554885916fd5ab402ab897`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:04.981919 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8efa00fa-2d7d-492f-b57b-ad29fd427377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:05.011367 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 817/1626] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:26:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 10:26:05.245166 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 810/1627] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:04 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:26:05.258034 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc1d010-e625-4277-89fc-683faafc6f7d service nova] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:05.484371 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc1d010-e625-4277-89fc-683faafc6f7d service nova] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:05.621575 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d6e24a-246f-4aad-b37a-df7242a7eb8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:05.622301 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d6e24a-246f-4aad-b37a-df7242a7eb8b service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:05.622804 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d6e24a-246f-4aad-b37a-df7242a7eb8b service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:05.623069 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d6e24a-246f-4aad-b37a-df7242a7eb8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:05.636451 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d6e24a-246f-4aad-b37a-df7242a7eb8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:05.639111 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d6e24a-246f-4aad-b37a-df7242a7eb8b service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:05.641542 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469ff03-8a02-4a73-abf8-10021ec095cc None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:05.642384 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469ff03-8a02-4a73-abf8-10021ec095cc None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:05.642729 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469ff03-8a02-4a73-abf8-10021ec095cc None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:05.645281 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 811/1628] 10.222.0.40 () {54 vars in 912 bytes} [Fri Aug 5 10:26:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:26:05.653946 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975df2f6-a70a-446b-b4bc-9790aa0480b6 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:05.654231 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975df2f6-a70a-446b-b4bc-9790aa0480b6 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:05.654548 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975df2f6-a70a-446b-b4bc-9790aa0480b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:05.673622 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:05.693821 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 818/1629] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:26:05 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:26:05.705136 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-975df2f6-a70a-446b-b4bc-9790aa0480b6 None None] MFA Rules not processed for user `b938f1f41db94b0eafe95bc8eeaae175`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:05.706536 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-975df2f6-a70a-446b-b4bc-9790aa0480b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:05.805467 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 812/1630] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 5 10:26:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:05.829522 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d6e24a-246f-4aad-b37a-df7242a7eb8b service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:05.843474 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d6e24a-246f-4aad-b37a-df7242a7eb8b service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:05.850050 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3310e0cc-91df-4107-9d60-d2ed515bc4cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:05.851382 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3310e0cc-91df-4107-9d60-d2ed515bc4cd service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:05.852757 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3310e0cc-91df-4107-9d60-d2ed515bc4cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:05.852757 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3310e0cc-91df-4107-9d60-d2ed515bc4cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:05.877459 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3310e0cc-91df-4107-9d60-d2ed515bc4cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:05.879419 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3310e0cc-91df-4107-9d60-d2ed515bc4cd service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:05.923105 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 819/1631] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:26:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 10:26:07.499531 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea47435-bc1a-4236-92e4-68169953cfdd None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:07.500010 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea47435-bc1a-4236-92e4-68169953cfdd None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:07.500218 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea47435-bc1a-4236-92e4-68169953cfdd None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:07.501865 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 813/1632] 10.222.0.40 () {54 vars in 962 bytes} [Fri Aug 5 10:26:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:26:07.510394 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e875286-c21a-4387-a2e1-9eb7041fb0ac None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:07.510976 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e875286-c21a-4387-a2e1-9eb7041fb0ac None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:07.510976 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e875286-c21a-4387-a2e1-9eb7041fb0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:07.556079 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8e875286-c21a-4387-a2e1-9eb7041fb0ac None None] MFA Rules not processed for user `b938f1f41db94b0eafe95bc8eeaae175`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:07.561439 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8e875286-c21a-4387-a2e1-9eb7041fb0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:07.609256 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 820/1633] 10.222.0.40 () {58 vars in 1059 bytes} [Fri Aug 5 10:26:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:07.616134 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642bb84-70c4-49ea-8784-2e6cb69f9646 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:07.616478 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642bb84-70c4-49ea-8784-2e6cb69f9646 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:07.616836 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642bb84-70c4-49ea-8784-2e6cb69f9646 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:07.619205 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 814/1634] 10.222.0.40 () {54 vars in 912 bytes} [Fri Aug 5 10:26:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:26:07.628027 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873068a-39cd-4b14-8c0a-1fa360b8a8e9 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:07.628313 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873068a-39cd-4b14-8c0a-1fa360b8a8e9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:07.628587 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873068a-39cd-4b14-8c0a-1fa360b8a8e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:07.654709 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-0873068a-39cd-4b14-8c0a-1fa360b8a8e9 None None] MFA Rules not processed for user `b938f1f41db94b0eafe95bc8eeaae175`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:07.656266 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0873068a-39cd-4b14-8c0a-1fa360b8a8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:07.687049 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 821/1635] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 5 10:26:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:26:07.738345 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4642bb84-70c4-49ea-8784-2e6cb69f9646 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:07.759946 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4642bb84-70c4-49ea-8784-2e6cb69f9646 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:07.773779 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d131e-5af8-4454-a363-97471613e835 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:07.775301 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d131e-5af8-4454-a363-97471613e835 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:07.777713 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d131e-5af8-4454-a363-97471613e835 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:07.778495 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d131e-5af8-4454-a363-97471613e835 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:07.813131 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1d131e-5af8-4454-a363-97471613e835 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:07.814767 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1d131e-5af8-4454-a363-97471613e835 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:07.853000 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 815/1636] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:26:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:26:09.395221 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0873068a-39cd-4b14-8c0a-1fa360b8a8e9 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:09.435384 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:09.490474 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0873068a-39cd-4b14-8c0a-1fa360b8a8e9 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:09.495486 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed33f384-788e-44e3-a054-453386e19522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:09.496482 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33f384-788e-44e3-a054-453386e19522 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:09.496900 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33f384-788e-44e3-a054-453386e19522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:09.497278 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33f384-788e-44e3-a054-453386e19522 admin admin] PATH_INFO: `/v3/users/805164588f414dd084eae660fd52eda3` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:09.500335 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed33f384-788e-44e3-a054-453386e19522 admin admin] RBAC: Authorizing `identity:delete_user(user_id=805164588f414dd084eae660fd52eda3)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:09.503458 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed33f384-788e-44e3-a054-453386e19522 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:09.591575 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ed33f384-788e-44e3-a054-453386e19522 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '805164588f414dd084eae660fd52eda3'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:09.592294 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ed33f384-788e-44e3-a054-453386e19522 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '805164588f414dd084eae660fd52eda3'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:09.593500 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ed33f384-788e-44e3-a054-453386e19522 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '805164588f414dd084eae660fd52eda3'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:09.618815 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ed33f384-788e-44e3-a054-453386e19522 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '805164588f414dd084eae660fd52eda3'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:09.630724 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ed33f384-788e-44e3-a054-453386e19522 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '805164588f414dd084eae660fd52eda3'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:09.700084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 822/1637] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:26:09 2022] DELETE /identity/v3/users/805164588f414dd084eae660fd52eda3 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:09.704413 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb9f6c-3d6a-4010-b22d-8ca7196b3a99 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:09.704912 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb9f6c-3d6a-4010-b22d-8ca7196b3a99 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:09.705255 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb9f6c-3d6a-4010-b22d-8ca7196b3a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:09.746321 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed33f384-788e-44e3-a054-453386e19522 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:09.747881 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-5dcb9f6c-3d6a-4010-b22d-8ca7196b3a99 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:09.748698 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ed33f384-788e-44e3-a054-453386e19522 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:09.749093 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5dcb9f6c-3d6a-4010-b22d-8ca7196b3a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:09.860808 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed33f384-788e-44e3-a054-453386e19522 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:09.868305 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb71af-5cb2-4118-9145-21911e2d180b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:09.869283 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb71af-5cb2-4118-9145-21911e2d180b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:09.870270 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb71af-5cb2-4118-9145-21911e2d180b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:09.870956 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb71af-5cb2-4118-9145-21911e2d180b admin admin] PATH_INFO: `/v3/users/3478f4fce4034422b8313310a834b364` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:09.873760 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fb71af-5cb2-4118-9145-21911e2d180b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3478f4fce4034422b8313310a834b364)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:09.875398 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fb71af-5cb2-4118-9145-21911e2d180b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:09.917831 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 816/1638] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:09.961951 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-21fb71af-5cb2-4118-9145-21911e2d180b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3478f4fce4034422b8313310a834b364'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:09.963698 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-21fb71af-5cb2-4118-9145-21911e2d180b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3478f4fce4034422b8313310a834b364'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:09.980407 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-21fb71af-5cb2-4118-9145-21911e2d180b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3478f4fce4034422b8313310a834b364'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:09.992244 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:10.014669 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-21fb71af-5cb2-4118-9145-21911e2d180b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3478f4fce4034422b8313310a834b364'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:10.039208 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-21fb71af-5cb2-4118-9145-21911e2d180b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3478f4fce4034422b8313310a834b364'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:10.087830 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 823/1639] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:26:09 2022] DELETE /identity/v3/users/3478f4fce4034422b8313310a834b364 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:10.093705 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fff557-ae1b-4282-a042-7640d001f2e9 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:10.093999 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fff557-ae1b-4282-a042-7640d001f2e9 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:10.094363 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fff557-ae1b-4282-a042-7640d001f2e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:10.131926 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a2fff557-ae1b-4282-a042-7640d001f2e9 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:10.133180 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a2fff557-ae1b-4282-a042-7640d001f2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:10.211136 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb71af-5cb2-4118-9145-21911e2d180b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:10.213618 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-21fb71af-5cb2-4118-9145-21911e2d180b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:10.318895 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 817/1640] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:10.345038 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb71af-5cb2-4118-9145-21911e2d180b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:10.355387 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea1dedf-d395-4c3d-98f5-0da32a3b7b8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:10.357251 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea1dedf-d395-4c3d-98f5-0da32a3b7b8b service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:10.357959 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea1dedf-d395-4c3d-98f5-0da32a3b7b8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:10.358435 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea1dedf-d395-4c3d-98f5-0da32a3b7b8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:10.363982 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fea1dedf-d395-4c3d-98f5-0da32a3b7b8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:10.393929 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea1dedf-d395-4c3d-98f5-0da32a3b7b8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:10.396822 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea1dedf-d395-4c3d-98f5-0da32a3b7b8b service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:10.405516 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fff557-ae1b-4282-a042-7640d001f2e9 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:10.422127 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fff557-ae1b-4282-a042-7640d001f2e9 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:10.438375 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327c36cb-7baa-4983-9a33-c932afbb8653 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:10.441369 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c36cb-7baa-4983-9a33-c932afbb8653 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:10.442766 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c36cb-7baa-4983-9a33-c932afbb8653 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:10.443461 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c36cb-7baa-4983-9a33-c932afbb8653 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:10.445136 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 824/1641] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:26:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:26:10.479264 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327c36cb-7baa-4983-9a33-c932afbb8653 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:10.489052 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327c36cb-7baa-4983-9a33-c932afbb8653 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:10.522435 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 818/1642] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:26:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 10:26:11.630412 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea1dedf-d395-4c3d-98f5-0da32a3b7b8b service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:11.633586 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fea1dedf-d395-4c3d-98f5-0da32a3b7b8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:11.654802 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea1dedf-d395-4c3d-98f5-0da32a3b7b8b service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:11.668300 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fb877e-7a98-4337-86df-5998e6054e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:11.668971 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb877e-7a98-4337-86df-5998e6054e94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:11.671826 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb877e-7a98-4337-86df-5998e6054e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:11.672159 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb877e-7a98-4337-86df-5998e6054e94 admin admin] PATH_INFO: `/v3/projects/59b286e5a3bc416f996a249eadb38c11` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:11.674968 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fb877e-7a98-4337-86df-5998e6054e94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59b286e5a3bc416f996a249eadb38c11)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:11.678605 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: WARNING py.warnings [None req-b7fb877e-7a98-4337-86df-5998e6054e94 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 10:26:11.678605 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: warnings.warn(msg) Aug 05 10:26:11.678605 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: Aug 05 10:26:11.682051 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fb877e-7a98-4337-86df-5998e6054e94 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:11.859938 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b7fb877e-7a98-4337-86df-5998e6054e94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59b286e5a3bc416f996a249eadb38c11', 'request_id': 'req-b7fb877e-7a98-4337-86df-5998e6054e94'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:11.885253 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b7fb877e-7a98-4337-86df-5998e6054e94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59b286e5a3bc416f996a249eadb38c11', 'request_id': 'req-b7fb877e-7a98-4337-86df-5998e6054e94'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:11.912975 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b7fb877e-7a98-4337-86df-5998e6054e94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59b286e5a3bc416f996a249eadb38c11', 'request_id': 'req-b7fb877e-7a98-4337-86df-5998e6054e94'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:11.965888 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b7fb877e-7a98-4337-86df-5998e6054e94 admin admin] The token cache is being invalidate because project 59b286e5a3bc416f996a249eadb38c11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:26:11.966553 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b7fb877e-7a98-4337-86df-5998e6054e94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:11.970592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b7fb877e-7a98-4337-86df-5998e6054e94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:11.975527 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 825/1643] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:26:11 2022] DELETE /identity/v3/projects/59b286e5a3bc416f996a249eadb38c11 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:11.979391 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327c36cb-7baa-4983-9a33-c932afbb8653 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:11.995052 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327c36cb-7baa-4983-9a33-c932afbb8653 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:12.008002 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:12.146964 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be682c1-af3a-418b-9046-8a003f136a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:12.147738 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be682c1-af3a-418b-9046-8a003f136a53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:12.148021 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be682c1-af3a-418b-9046-8a003f136a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:12.148291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be682c1-af3a-418b-9046-8a003f136a53 admin admin] PATH_INFO: `/v3/users/28c01479ae554885916fd5ab402ab897` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:12.175703 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be682c1-af3a-418b-9046-8a003f136a53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28c01479ae554885916fd5ab402ab897)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:12.176898 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: WARNING py.warnings [None req-4be682c1-af3a-418b-9046-8a003f136a53 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 10:26:12.176898 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: warnings.warn(msg) Aug 05 10:26:12.176898 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: Aug 05 10:26:12.178249 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be682c1-af3a-418b-9046-8a003f136a53 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:12.407969 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4be682c1-af3a-418b-9046-8a003f136a53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28c01479ae554885916fd5ab402ab897'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:12.464600 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4be682c1-af3a-418b-9046-8a003f136a53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28c01479ae554885916fd5ab402ab897'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:12.467709 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4be682c1-af3a-418b-9046-8a003f136a53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28c01479ae554885916fd5ab402ab897'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:12.528210 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4be682c1-af3a-418b-9046-8a003f136a53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28c01479ae554885916fd5ab402ab897'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:12.532600 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4be682c1-af3a-418b-9046-8a003f136a53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28c01479ae554885916fd5ab402ab897'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:12.567992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fb877e-7a98-4337-86df-5998e6054e94 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:12.570472 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b7fb877e-7a98-4337-86df-5998e6054e94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:12.575827 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 819/1644] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:26:11 2022] DELETE /identity/v3/users/28c01479ae554885916fd5ab402ab897 => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:12.603932 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fb877e-7a98-4337-86df-5998e6054e94 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:12.642249 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a0869-d5e8-4213-b894-60bc661984c7 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:12.642596 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a0869-d5e8-4213-b894-60bc661984c7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:12.642866 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a0869-d5e8-4213-b894-60bc661984c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:12.735542 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b72a0869-d5e8-4213-b894-60bc661984c7 None None] MFA Rules not processed for user `706dbfa719c04fd0a3704ebd7eb812cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:12.737409 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b72a0869-d5e8-4213-b894-60bc661984c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:12.793595 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:12.821723 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf64053-e6e8-4fca-b160-7579be164646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:12.822482 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf64053-e6e8-4fca-b160-7579be164646 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:12.823079 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf64053-e6e8-4fca-b160-7579be164646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:12.823369 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf64053-e6e8-4fca-b160-7579be164646 admin admin] PATH_INFO: `/v3/projects/78c1e92af310437db4b79bfed67fa76a` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:12.824707 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf64053-e6e8-4fca-b160-7579be164646 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78c1e92af310437db4b79bfed67fa76a)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:12.827001 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf64053-e6e8-4fca-b160-7579be164646 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:12.951161 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7bf64053-e6e8-4fca-b160-7579be164646 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78c1e92af310437db4b79bfed67fa76a', 'request_id': 'req-7bf64053-e6e8-4fca-b160-7579be164646'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:12.964946 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 820/1645] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:26:12.981057 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a0869-d5e8-4213-b894-60bc661984c7 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:12.982739 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b72a0869-d5e8-4213-b894-60bc661984c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:12.984724 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7bf64053-e6e8-4fca-b160-7579be164646 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78c1e92af310437db4b79bfed67fa76a', 'request_id': 'req-7bf64053-e6e8-4fca-b160-7579be164646'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:13.000818 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7bf64053-e6e8-4fca-b160-7579be164646 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78c1e92af310437db4b79bfed67fa76a', 'request_id': 'req-7bf64053-e6e8-4fca-b160-7579be164646'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:13.022520 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7bf64053-e6e8-4fca-b160-7579be164646 admin admin] The token cache is being invalidate because project 78c1e92af310437db4b79bfed67fa76a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:26:13.023636 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7bf64053-e6e8-4fca-b160-7579be164646 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:13.023636 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7bf64053-e6e8-4fca-b160-7579be164646 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:13.025462 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 826/1646] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:26:12 2022] DELETE /identity/v3/projects/78c1e92af310437db4b79bfed67fa76a => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:13.030514 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc68f7-7e9a-4f0c-bc79-cd9e69d32525 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:13.032226 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc68f7-7e9a-4f0c-bc79-cd9e69d32525 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:13.032645 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc68f7-7e9a-4f0c-bc79-cd9e69d32525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:13.034734 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:13.085472 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-40bc68f7-7e9a-4f0c-bc79-cd9e69d32525 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:13.089648 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-40bc68f7-7e9a-4f0c-bc79-cd9e69d32525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:13.278395 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a0869-d5e8-4213-b894-60bc661984c7 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:13.307119 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 827/1647] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:13.312651 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717936ec-76d6-4060-af2a-677722112ae7 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:13.313027 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717936ec-76d6-4060-af2a-677722112ae7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:13.313458 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717936ec-76d6-4060-af2a-677722112ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:13.370038 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-717936ec-76d6-4060-af2a-677722112ae7 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:13.377420 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-717936ec-76d6-4060-af2a-677722112ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:13.398282 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 828/1648] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:13.403494 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717936ec-76d6-4060-af2a-677722112ae7 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:13.428878 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717936ec-76d6-4060-af2a-677722112ae7 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:13.451572 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d3f70f-2c11-4ce7-ae2f-c4a940199860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:13.452224 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d3f70f-2c11-4ce7-ae2f-c4a940199860 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:13.452507 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d3f70f-2c11-4ce7-ae2f-c4a940199860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:13.453648 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d3f70f-2c11-4ce7-ae2f-c4a940199860 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:13.454311 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d3f70f-2c11-4ce7-ae2f-c4a940199860 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:26:13.454619 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d3f70f-2c11-4ce7-ae2f-c4a940199860 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:13.456745 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d3f70f-2c11-4ce7-ae2f-c4a940199860 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:13.468238 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6caad07-0a1d-4e26-8eab-ee0a00b076a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:13.469146 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6caad07-0a1d-4e26-8eab-ee0a00b076a4 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:13.475042 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6caad07-0a1d-4e26-8eab-ee0a00b076a4 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:13.475406 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6caad07-0a1d-4e26-8eab-ee0a00b076a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:13.485620 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a6caad07-0a1d-4e26-8eab-ee0a00b076a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:13.511807 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 829/1649] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:26:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:26:13.512748 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d3f70f-2c11-4ce7-ae2f-c4a940199860 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:13.543089 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d3f70f-2c11-4ce7-ae2f-c4a940199860 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:13.555344 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecd5b88-882a-4a4b-be2a-faaa0c43f50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:13.556906 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd5b88-882a-4a4b-be2a-faaa0c43f50a admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:13.557203 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd5b88-882a-4a4b-be2a-faaa0c43f50a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:13.558235 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd5b88-882a-4a4b-be2a-faaa0c43f50a admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:13.559438 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecd5b88-882a-4a4b-be2a-faaa0c43f50a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:13.565098 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecd5b88-882a-4a4b-be2a-faaa0c43f50a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:13.614337 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 830/1650] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:26:13 2022] POST /identity/v3/projects => generated 405 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:26:13.621538 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecd5b88-882a-4a4b-be2a-faaa0c43f50a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:13.626281 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0ecd5b88-882a-4a4b-be2a-faaa0c43f50a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:13.669476 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6caad07-0a1d-4e26-8eab-ee0a00b076a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:13.671880 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6caad07-0a1d-4e26-8eab-ee0a00b076a4 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:13.693895 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 821/1651] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:26:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 714 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:26:13.695623 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6caad07-0a1d-4e26-8eab-ee0a00b076a4 service nova] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:13.714738 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6caad07-0a1d-4e26-8eab-ee0a00b076a4 service nova] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:13.748552 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:13.803399 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecd5b88-882a-4a4b-be2a-faaa0c43f50a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:13.816493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd92d07-6da7-415d-b24a-887b73a077e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:13.818163 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd92d07-6da7-415d-b24a-887b73a077e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:13.818451 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd92d07-6da7-415d-b24a-887b73a077e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:13.825001 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd92d07-6da7-415d-b24a-887b73a077e5 admin admin] PATH_INFO: `/v3/projects/d806e1db3b3b4db0adc10ded22256192` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:13.828443 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd92d07-6da7-415d-b24a-887b73a077e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d806e1db3b3b4db0adc10ded22256192)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:13.833644 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd92d07-6da7-415d-b24a-887b73a077e5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:13.840412 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:13.858767 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5749f2-e896-44af-b512-fc515447832d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:13.861685 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5749f2-e896-44af-b512-fc515447832d admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:13.863599 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5749f2-e896-44af-b512-fc515447832d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:13.864039 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5749f2-e896-44af-b512-fc515447832d admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:13.864892 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5749f2-e896-44af-b512-fc515447832d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:13.870802 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5749f2-e896-44af-b512-fc515447832d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:13.938839 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 822/1652] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:26:13 2022] POST /identity/v3/users => generated 303 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:26:13.944965 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5749f2-e896-44af-b512-fc515447832d admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:13.986613 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5749f2-e896-44af-b512-fc515447832d admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:13.991736 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-abd92d07-6da7-415d-b24a-887b73a077e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd806e1db3b3b4db0adc10ded22256192', 'request_id': 'req-abd92d07-6da7-415d-b24a-887b73a077e5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:14.019938 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-abd92d07-6da7-415d-b24a-887b73a077e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd806e1db3b3b4db0adc10ded22256192', 'request_id': 'req-abd92d07-6da7-415d-b24a-887b73a077e5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:14.043930 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-abd92d07-6da7-415d-b24a-887b73a077e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd806e1db3b3b4db0adc10ded22256192', 'request_id': 'req-abd92d07-6da7-415d-b24a-887b73a077e5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:14.088736 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-abd92d07-6da7-415d-b24a-887b73a077e5 admin admin] The token cache is being invalidate because project d806e1db3b3b4db0adc10ded22256192 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:26:14.089632 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-abd92d07-6da7-415d-b24a-887b73a077e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:14.096490 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-abd92d07-6da7-415d-b24a-887b73a077e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:14.107191 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 831/1653] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:26:13 2022] DELETE /identity/v3/projects/d806e1db3b3b4db0adc10ded22256192 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:14.109406 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea91fe-041d-45b0-91e4-a52c23004d88 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:14.109696 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea91fe-041d-45b0-91e4-a52c23004d88 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:14.109978 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea91fe-041d-45b0-91e4-a52c23004d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:14.189427 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-7eea91fe-041d-45b0-91e4-a52c23004d88 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:14.192595 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7eea91fe-041d-45b0-91e4-a52c23004d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:14.250639 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3424ec0-ae64-46dc-82b6-23e454144de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:14.251357 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3424ec0-ae64-46dc-82b6-23e454144de6 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:14.251644 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3424ec0-ae64-46dc-82b6-23e454144de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:14.251983 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3424ec0-ae64-46dc-82b6-23e454144de6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:14.260020 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3424ec0-ae64-46dc-82b6-23e454144de6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:14.261698 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3424ec0-ae64-46dc-82b6-23e454144de6 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:14.290278 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 823/1654] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 10:26:14.295087 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3424ec0-ae64-46dc-82b6-23e454144de6 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:14.302863 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d3424ec0-ae64-46dc-82b6-23e454144de6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:14.350808 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3424ec0-ae64-46dc-82b6-23e454144de6 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:14.368004 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7c77a7-0a0e-4930-a031-5a969cbc5fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:14.372616 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c77a7-0a0e-4930-a031-5a969cbc5fa2 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:14.374538 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c77a7-0a0e-4930-a031-5a969cbc5fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:14.374842 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c77a7-0a0e-4930-a031-5a969cbc5fa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:14.376680 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7c77a7-0a0e-4930-a031-5a969cbc5fa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:14.383628 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7c77a7-0a0e-4930-a031-5a969cbc5fa2 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:14.396617 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 832/1655] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:14.406331 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13969a5-700f-44f1-b7a7-c33b880328db None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:14.406641 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13969a5-700f-44f1-b7a7-c33b880328db None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:14.406886 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13969a5-700f-44f1-b7a7-c33b880328db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:14.425946 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 824/1656] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:14.426880 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7c77a7-0a0e-4930-a031-5a969cbc5fa2 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:14.452879 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e13969a5-700f-44f1-b7a7-c33b880328db None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:14.456862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e13969a5-700f-44f1-b7a7-c33b880328db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:14.463465 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7c77a7-0a0e-4930-a031-5a969cbc5fa2 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:14.474164 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edade53-6615-498b-a21d-6425935bbe95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:14.474938 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edade53-6615-498b-a21d-6425935bbe95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:14.475346 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edade53-6615-498b-a21d-6425935bbe95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:14.475637 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edade53-6615-498b-a21d-6425935bbe95 admin admin] PATH_INFO: `/v3/projects/c8cb7b6044e0420283e0cb1abbbb1ed2/users/dc2c9f055929440d8e9645cdf9bebec8/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:14.484119 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 833/1657] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:14.485631 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13969a5-700f-44f1-b7a7-c33b880328db None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:14.505184 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13969a5-700f-44f1-b7a7-c33b880328db None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:14.524232 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ffddd1-39ee-4cd0-89e5-79894347f504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:14.524881 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ffddd1-39ee-4cd0-89e5-79894347f504 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:14.525159 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ffddd1-39ee-4cd0-89e5-79894347f504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:14.525427 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ffddd1-39ee-4cd0-89e5-79894347f504 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:14.526130 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ffddd1-39ee-4cd0-89e5-79894347f504 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:26:14.526432 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ffddd1-39ee-4cd0-89e5-79894347f504 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:14.528085 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ffddd1-39ee-4cd0-89e5-79894347f504 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:14.529903 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edade53-6615-498b-a21d-6425935bbe95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8cb7b6044e0420283e0cb1abbbb1ed2, user_id=dc2c9f055929440d8e9645cdf9bebec8, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:14.535606 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edade53-6615-498b-a21d-6425935bbe95 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:14.559026 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 834/1658] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:26:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:26:14.559991 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ffddd1-39ee-4cd0-89e5-79894347f504 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:14.587670 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 825/1659] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:26:14 2022] PUT /identity/v3/projects/c8cb7b6044e0420283e0cb1abbbb1ed2/users/dc2c9f055929440d8e9645cdf9bebec8/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:14.589939 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ffddd1-39ee-4cd0-89e5-79894347f504 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:14.590643 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97234fe3-4237-41d8-89c7-d2405eb1fc7e None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:14.590884 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97234fe3-4237-41d8-89c7-d2405eb1fc7e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:14.591219 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97234fe3-4237-41d8-89c7-d2405eb1fc7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:14.626355 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-97234fe3-4237-41d8-89c7-d2405eb1fc7e None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:14.627820 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-97234fe3-4237-41d8-89c7-d2405eb1fc7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:14.707802 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:14.730634 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b1aa8-b37f-489b-b7c7-316813701bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:14.731384 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b1aa8-b37f-489b-b7c7-316813701bd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:14.731668 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b1aa8-b37f-489b-b7c7-316813701bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:14.731936 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b1aa8-b37f-489b-b7c7-316813701bd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:14.733979 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27b1aa8-b37f-489b-b7c7-316813701bd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:14.740471 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27b1aa8-b37f-489b-b7c7-316813701bd7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:14.760631 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 826/1660] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:14.813891 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 835/1661] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:26:14 2022] POST /identity/v3/projects => generated 393 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:26:14.817898 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97234fe3-4237-41d8-89c7-d2405eb1fc7e None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:14.831136 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97234fe3-4237-41d8-89c7-d2405eb1fc7e None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:14.946095 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:15.012446 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123e5f15-4b6e-4f3f-9702-d1096819016f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:15.015183 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e5f15-4b6e-4f3f-9702-d1096819016f admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:15.016511 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e5f15-4b6e-4f3f-9702-d1096819016f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:15.017064 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e5f15-4b6e-4f3f-9702-d1096819016f admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:15.017834 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123e5f15-4b6e-4f3f-9702-d1096819016f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:15.025707 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123e5f15-4b6e-4f3f-9702-d1096819016f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:15.082534 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 827/1662] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:26:14 2022] POST /identity/v3/users => generated 297 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:26:15.085490 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b1aa8-b37f-489b-b7c7-316813701bd7 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:15.103260 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b1aa8-b37f-489b-b7c7-316813701bd7 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:15.113461 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f046135-f66b-4564-bb50-64a84adf0719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:15.114296 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f046135-f66b-4564-bb50-64a84adf0719 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:15.114684 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f046135-f66b-4564-bb50-64a84adf0719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:15.115228 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f046135-f66b-4564-bb50-64a84adf0719 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:15.116052 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f046135-f66b-4564-bb50-64a84adf0719 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:15.119148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f046135-f66b-4564-bb50-64a84adf0719 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:15.144696 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 836/1663] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:15 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:15.145741 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123e5f15-4b6e-4f3f-9702-d1096819016f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:15.184523 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f046135-f66b-4564-bb50-64a84adf0719 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:15.186726 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123e5f15-4b6e-4f3f-9702-d1096819016f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:15.187219 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7f046135-f66b-4564-bb50-64a84adf0719 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:15.204058 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fcdcea-6d5c-4971-9823-5b4b18545f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:15.205063 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fcdcea-6d5c-4971-9823-5b4b18545f8d admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:15.205189 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fcdcea-6d5c-4971-9823-5b4b18545f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:15.205425 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fcdcea-6d5c-4971-9823-5b4b18545f8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:15.206357 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fcdcea-6d5c-4971-9823-5b4b18545f8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:15.207891 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fcdcea-6d5c-4971-9823-5b4b18545f8d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:15.264383 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 828/1664] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:15 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 10:26:15.265663 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fcdcea-6d5c-4971-9823-5b4b18545f8d admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:15.300613 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fcdcea-6d5c-4971-9823-5b4b18545f8d admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:15.316668 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab3ff64-1e03-4d77-a68d-6e23666d322d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:15.319177 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3ff64-1e03-4d77-a68d-6e23666d322d admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:15.319577 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3ff64-1e03-4d77-a68d-6e23666d322d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:15.320037 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3ff64-1e03-4d77-a68d-6e23666d322d admin admin] PATH_INFO: `/v3/projects/3e57f4b8241c4650940c57d675e39fab/users/318d06f754a4443eb20a3184a3d27d61/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:15.349020 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f046135-f66b-4564-bb50-64a84adf0719 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:15.359528 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab3ff64-1e03-4d77-a68d-6e23666d322d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e57f4b8241c4650940c57d675e39fab, user_id=318d06f754a4443eb20a3184a3d27d61, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:15.361932 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a9a08-3ea5-4f8d-97e7-0d13b51ae058 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:15.363313 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a9a08-3ea5-4f8d-97e7-0d13b51ae058 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:15.363598 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a9a08-3ea5-4f8d-97e7-0d13b51ae058 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:15.363940 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a9a08-3ea5-4f8d-97e7-0d13b51ae058 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:15.365678 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab3ff64-1e03-4d77-a68d-6e23666d322d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:15.388977 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65a9a08-3ea5-4f8d-97e7-0d13b51ae058 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:15.390533 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65a9a08-3ea5-4f8d-97e7-0d13b51ae058 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:15.402392 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 829/1665] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:26:15 2022] PUT /identity/v3/projects/3e57f4b8241c4650940c57d675e39fab/users/318d06f754a4443eb20a3184a3d27d61/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:15.426241 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 837/1666] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:26:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:26:15.428860 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33665d23-cb5d-442d-a087-b2998b2f1d0e None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:15.429361 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33665d23-cb5d-442d-a087-b2998b2f1d0e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:15.429790 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33665d23-cb5d-442d-a087-b2998b2f1d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:15.507377 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-33665d23-cb5d-442d-a087-b2998b2f1d0e None None] MFA Rules not processed for user `318d06f754a4443eb20a3184a3d27d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:15.508446 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-33665d23-cb5d-442d-a087-b2998b2f1d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:15.647105 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 830/1667] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:15 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:15.652035 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a9a08-3ea5-4f8d-97e7-0d13b51ae058 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:15.669005 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a9a08-3ea5-4f8d-97e7-0d13b51ae058 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:15.782550 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf0488-3eff-4ef7-bc4b-c349e169ed84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:15.783906 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf0488-3eff-4ef7-bc4b-c349e169ed84 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:15.785636 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf0488-3eff-4ef7-bc4b-c349e169ed84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:15.785970 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf0488-3eff-4ef7-bc4b-c349e169ed84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:15.788403 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdf0488-3eff-4ef7-bc4b-c349e169ed84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:15.792933 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdf0488-3eff-4ef7-bc4b-c349e169ed84 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:15.846954 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 838/1668] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:26:15 2022] POST /identity/v3/projects => generated 391 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:26:15.850773 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33665d23-cb5d-442d-a087-b2998b2f1d0e None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:15.865926 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33665d23-cb5d-442d-a087-b2998b2f1d0e None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:15.985372 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1d9927-0cc8-46ac-9ea2-9e469b101564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:15.986292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d9927-0cc8-46ac-9ea2-9e469b101564 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:15.986924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d9927-0cc8-46ac-9ea2-9e469b101564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:15.987396 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d9927-0cc8-46ac-9ea2-9e469b101564 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:15.988479 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1d9927-0cc8-46ac-9ea2-9e469b101564 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:15.990953 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1d9927-0cc8-46ac-9ea2-9e469b101564 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:16.027017 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 831/1669] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:26:15 2022] POST /identity/v3/users => generated 302 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:26:16.031658 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf0488-3eff-4ef7-bc4b-c349e169ed84 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:16.053133 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf0488-3eff-4ef7-bc4b-c349e169ed84 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:16.074725 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87e0805-fe29-4518-8015-6fd4db13fc69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:16.075413 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e0805-fe29-4518-8015-6fd4db13fc69 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:16.075694 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e0805-fe29-4518-8015-6fd4db13fc69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:16.075966 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e0805-fe29-4518-8015-6fd4db13fc69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:16.076723 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87e0805-fe29-4518-8015-6fd4db13fc69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:16.078480 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87e0805-fe29-4518-8015-6fd4db13fc69 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:16.103111 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 839/1670] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:16 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:16.107734 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1d9927-0cc8-46ac-9ea2-9e469b101564 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:16.120954 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1d9927-0cc8-46ac-9ea2-9e469b101564 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:16.126446 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248df0f9-d543-4ff8-a255-df4f2203661c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:16.127115 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248df0f9-d543-4ff8-a255-df4f2203661c admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:16.127493 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248df0f9-d543-4ff8-a255-df4f2203661c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:16.127874 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248df0f9-d543-4ff8-a255-df4f2203661c admin admin] PATH_INFO: `/v3/projects/bd69bbc96ce349f3a9cd44669c50c9e3/users/e74abcfb8cde47d18ecec1ded868d2ed/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:16.160301 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248df0f9-d543-4ff8-a255-df4f2203661c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd69bbc96ce349f3a9cd44669c50c9e3, user_id=e74abcfb8cde47d18ecec1ded868d2ed, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:16.165822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248df0f9-d543-4ff8-a255-df4f2203661c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:16.171315 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:16.212717 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 832/1671] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:26:16 2022] PUT /identity/v3/projects/bd69bbc96ce349f3a9cd44669c50c9e3/users/e74abcfb8cde47d18ecec1ded868d2ed/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:16.216897 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87e0805-fe29-4518-8015-6fd4db13fc69 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:16.239585 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87e0805-fe29-4518-8015-6fd4db13fc69 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:16.352446 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:16.397249 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcddc75b-e346-4cac-830c-19c5369c633c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:16.400291 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcddc75b-e346-4cac-830c-19c5369c633c admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:16.400579 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcddc75b-e346-4cac-830c-19c5369c633c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:16.402637 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcddc75b-e346-4cac-830c-19c5369c633c admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:16.407500 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcddc75b-e346-4cac-830c-19c5369c633c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:16.413207 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcddc75b-e346-4cac-830c-19c5369c633c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:16.445879 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 840/1672] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:16 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:16.451099 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248df0f9-d543-4ff8-a255-df4f2203661c admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:16.468717 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248df0f9-d543-4ff8-a255-df4f2203661c admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:16.476866 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5234e73-2cd0-4f2d-91e7-2a8fa0fc3371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:16.477579 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5234e73-2cd0-4f2d-91e7-2a8fa0fc3371 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:16.477861 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5234e73-2cd0-4f2d-91e7-2a8fa0fc3371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:16.478186 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5234e73-2cd0-4f2d-91e7-2a8fa0fc3371 admin admin] PATH_INFO: `/v3/projects/bd69bbc96ce349f3a9cd44669c50c9e3/users/e74abcfb8cde47d18ecec1ded868d2ed/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:16.482328 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5234e73-2cd0-4f2d-91e7-2a8fa0fc3371 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd69bbc96ce349f3a9cd44669c50c9e3, user_id=e74abcfb8cde47d18ecec1ded868d2ed, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:16.488424 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5234e73-2cd0-4f2d-91e7-2a8fa0fc3371 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:16.512322 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 833/1673] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:26:16 2022] PUT /identity/v3/projects/bd69bbc96ce349f3a9cd44669c50c9e3/users/e74abcfb8cde47d18ecec1ded868d2ed/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:26:16.522626 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f368db1-7613-4057-a6b5-fa0710b4f0b2 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:16.523762 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f368db1-7613-4057-a6b5-fa0710b4f0b2 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:16.524125 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f368db1-7613-4057-a6b5-fa0710b4f0b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:16.611460 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1f368db1-7613-4057-a6b5-fa0710b4f0b2 None None] MFA Rules not processed for user `e74abcfb8cde47d18ecec1ded868d2ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:16.613997 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1f368db1-7613-4057-a6b5-fa0710b4f0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:16.776749 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 841/1674] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:16 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:16.798824 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5234e73-2cd0-4f2d-91e7-2a8fa0fc3371 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:16.800915 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d5234e73-2cd0-4f2d-91e7-2a8fa0fc3371 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:16.884534 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5234e73-2cd0-4f2d-91e7-2a8fa0fc3371 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:16.889057 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908f754-259c-4a7c-9f4f-8fa1c79781e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:16.889697 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908f754-259c-4a7c-9f4f-8fa1c79781e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:16.889977 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908f754-259c-4a7c-9f4f-8fa1c79781e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:16.890247 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908f754-259c-4a7c-9f4f-8fa1c79781e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:16.904645 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1908f754-259c-4a7c-9f4f-8fa1c79781e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:16.905816 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1908f754-259c-4a7c-9f4f-8fa1c79781e7 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:16.931148 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 834/1675] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:26:16 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:26:17.009513 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f368db1-7613-4057-a6b5-fa0710b4f0b2 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:17.012622 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1f368db1-7613-4057-a6b5-fa0710b4f0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:17.029593 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f368db1-7613-4057-a6b5-fa0710b4f0b2 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:17.033882 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f2680-5410-4439-94b5-55c8d68e9ed8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:17.034560 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f2680-5410-4439-94b5-55c8d68e9ed8 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:17.034882 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f2680-5410-4439-94b5-55c8d68e9ed8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:17.035400 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f2680-5410-4439-94b5-55c8d68e9ed8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:17.174182 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f2680-5410-4439-94b5-55c8d68e9ed8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:17.175760 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f2680-5410-4439-94b5-55c8d68e9ed8 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:17.191849 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 842/1676] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:26:17 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:26:17.329402 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908f754-259c-4a7c-9f4f-8fa1c79781e7 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:17.332274 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1908f754-259c-4a7c-9f4f-8fa1c79781e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:17.540456 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908f754-259c-4a7c-9f4f-8fa1c79781e7 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:17.553922 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2764ee86-e50f-4aba-b883-e5834dbf5a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:17.554890 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764ee86-e50f-4aba-b883-e5834dbf5a82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:17.555205 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764ee86-e50f-4aba-b883-e5834dbf5a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:17.555477 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764ee86-e50f-4aba-b883-e5834dbf5a82 admin admin] PATH_INFO: `/v3/users/20eb8a3abe034044a1785f6b7f6a5d4b` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:17.557840 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2764ee86-e50f-4aba-b883-e5834dbf5a82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20eb8a3abe034044a1785f6b7f6a5d4b)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:17.560429 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2764ee86-e50f-4aba-b883-e5834dbf5a82 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:17.679505 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2764ee86-e50f-4aba-b883-e5834dbf5a82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20eb8a3abe034044a1785f6b7f6a5d4b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:17.696917 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2764ee86-e50f-4aba-b883-e5834dbf5a82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20eb8a3abe034044a1785f6b7f6a5d4b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:17.697707 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2764ee86-e50f-4aba-b883-e5834dbf5a82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20eb8a3abe034044a1785f6b7f6a5d4b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:17.725542 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2764ee86-e50f-4aba-b883-e5834dbf5a82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20eb8a3abe034044a1785f6b7f6a5d4b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:17.726508 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2764ee86-e50f-4aba-b883-e5834dbf5a82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20eb8a3abe034044a1785f6b7f6a5d4b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:17.752943 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 835/1677] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:26:17 2022] DELETE /identity/v3/users/20eb8a3abe034044a1785f6b7f6a5d4b => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:17.758357 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bee0e3-0d6b-4e4c-b61b-e6564f09e98b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:17.759056 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bee0e3-0d6b-4e4c-b61b-e6564f09e98b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:17.759605 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bee0e3-0d6b-4e4c-b61b-e6564f09e98b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:17.806647 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-41bee0e3-0d6b-4e4c-b61b-e6564f09e98b None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:17.809860 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-41bee0e3-0d6b-4e4c-b61b-e6564f09e98b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:17.962126 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 843/1678] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:18.075927 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2764ee86-e50f-4aba-b883-e5834dbf5a82 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:18.078695 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2764ee86-e50f-4aba-b883-e5834dbf5a82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:18.086449 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:18.178884 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2764ee86-e50f-4aba-b883-e5834dbf5a82 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:18.183197 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d152cb-dd8d-4432-8d33-0b4f5d7082b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:18.184425 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d152cb-dd8d-4432-8d33-0b4f5d7082b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:18.186355 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d152cb-dd8d-4432-8d33-0b4f5d7082b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:18.186880 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d152cb-dd8d-4432-8d33-0b4f5d7082b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:18.200275 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d152cb-dd8d-4432-8d33-0b4f5d7082b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:18.202689 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d152cb-dd8d-4432-8d33-0b4f5d7082b4 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:18.225194 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 836/1679] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:26:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:26:19.282523 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4527ae0-5355-49f8-a13f-bb162c5b5870 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:19.283481 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4527ae0-5355-49f8-a13f-bb162c5b5870 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:19.283932 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4527ae0-5355-49f8-a13f-bb162c5b5870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:19.296681 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:19.341154 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d4527ae0-5355-49f8-a13f-bb162c5b5870 None None] MFA Rules not processed for user `a3b4d90bcfc34d2d9ba1c40316e09858`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:19.342486 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d4527ae0-5355-49f8-a13f-bb162c5b5870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:19.485492 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 844/1680] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:19 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:19.497022 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e48eff-4597-4e90-98e9-19b38f23bc06 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:19.497613 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e48eff-4597-4e90-98e9-19b38f23bc06 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:19.497613 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e48eff-4597-4e90-98e9-19b38f23bc06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:19.522804 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-06e48eff-4597-4e90-98e9-19b38f23bc06 None None] MFA Rules not processed for user `a3b4d90bcfc34d2d9ba1c40316e09858`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:19.524120 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-06e48eff-4597-4e90-98e9-19b38f23bc06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:19.557436 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 837/1681] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:19 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:19.579104 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4527ae0-5355-49f8-a13f-bb162c5b5870 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:19.580769 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d4527ae0-5355-49f8-a13f-bb162c5b5870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:19.811871 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3122da-8f4c-4fc1-abd5-c4dfbc6ca040 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:19.812292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3122da-8f4c-4fc1-abd5-c4dfbc6ca040 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:19.812659 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3122da-8f4c-4fc1-abd5-c4dfbc6ca040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:19.840999 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4527ae0-5355-49f8-a13f-bb162c5b5870 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:19.849040 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de979625-a9c1-489f-9751-25127c602e8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:19.850068 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de979625-a9c1-489f-9751-25127c602e8a service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:19.850501 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de979625-a9c1-489f-9751-25127c602e8a service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:19.851020 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de979625-a9c1-489f-9751-25127c602e8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:19.856097 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ca3122da-8f4c-4fc1-abd5-c4dfbc6ca040 None None] MFA Rules not processed for user `706dbfa719c04fd0a3704ebd7eb812cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:19.857515 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ca3122da-8f4c-4fc1-abd5-c4dfbc6ca040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:19.863526 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de979625-a9c1-489f-9751-25127c602e8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:19.865876 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de979625-a9c1-489f-9751-25127c602e8a service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:19.884874 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 845/1682] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:26:19 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 10:26:19.886286 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de979625-a9c1-489f-9751-25127c602e8a service nova] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:19.887892 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-de979625-a9c1-489f-9751-25127c602e8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:19.912664 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de979625-a9c1-489f-9751-25127c602e8a service nova] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:19.928402 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4fc92-6f20-4b2e-b365-ef766505bc09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:19.931120 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4fc92-6f20-4b2e-b365-ef766505bc09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:19.931435 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4fc92-6f20-4b2e-b365-ef766505bc09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:19.933635 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4fc92-6f20-4b2e-b365-ef766505bc09 admin admin] PATH_INFO: `/v3/projects/a33f145210734116a61371c8e6ba404a` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:19.936563 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde4fc92-6f20-4b2e-b365-ef766505bc09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a33f145210734116a61371c8e6ba404a)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:19.948916 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde4fc92-6f20-4b2e-b365-ef766505bc09 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:20.048749 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fde4fc92-6f20-4b2e-b365-ef766505bc09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a33f145210734116a61371c8e6ba404a', 'request_id': 'req-fde4fc92-6f20-4b2e-b365-ef766505bc09'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:20.066331 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fde4fc92-6f20-4b2e-b365-ef766505bc09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a33f145210734116a61371c8e6ba404a', 'request_id': 'req-fde4fc92-6f20-4b2e-b365-ef766505bc09'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:20.072357 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 838/1683] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:20.093200 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fde4fc92-6f20-4b2e-b365-ef766505bc09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a33f145210734116a61371c8e6ba404a', 'request_id': 'req-fde4fc92-6f20-4b2e-b365-ef766505bc09'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:20.130558 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:20.149079 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fde4fc92-6f20-4b2e-b365-ef766505bc09 admin admin] The token cache is being invalidate because project a33f145210734116a61371c8e6ba404a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:26:20.149687 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fde4fc92-6f20-4b2e-b365-ef766505bc09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:20.162433 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fde4fc92-6f20-4b2e-b365-ef766505bc09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:20.166135 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 846/1684] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:26:19 2022] DELETE /identity/v3/projects/a33f145210734116a61371c8e6ba404a => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:20.171262 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb9c67-4ea2-47f0-91b6-da8140933aeb None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:20.171757 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb9c67-4ea2-47f0-91b6-da8140933aeb None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:20.172104 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb9c67-4ea2-47f0-91b6-da8140933aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:20.219138 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-facb9c67-4ea2-47f0-91b6-da8140933aeb None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:20.222419 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-facb9c67-4ea2-47f0-91b6-da8140933aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:20.286019 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fdf95d-6304-4966-b1e3-197e7250362b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:20.287259 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fdf95d-6304-4966-b1e3-197e7250362b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:20.287700 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fdf95d-6304-4966-b1e3-197e7250362b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:20.396535 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 839/1685] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:20.405753 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22f33b-8fc9-4384-9b2f-f3e91c715de1 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:20.406144 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22f33b-8fc9-4384-9b2f-f3e91c715de1 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:20.409005 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22f33b-8fc9-4384-9b2f-f3e91c715de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:20.411018 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-84fdf95d-6304-4966-b1e3-197e7250362b None None] MFA Rules not processed for user `dc2c9f055929440d8e9645cdf9bebec8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:20.412699 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-84fdf95d-6304-4966-b1e3-197e7250362b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:20.456583 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-0b22f33b-8fc9-4384-9b2f-f3e91c715de1 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:20.456583 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0b22f33b-8fc9-4384-9b2f-f3e91c715de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:20.503463 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 840/1686] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:20.507910 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b22f33b-8fc9-4384-9b2f-f3e91c715de1 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:20.529740 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b22f33b-8fc9-4384-9b2f-f3e91c715de1 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:20.556065 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d23f97b-5764-4e14-9988-f43f784c1f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:20.557151 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23f97b-5764-4e14-9988-f43f784c1f2f admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:20.557151 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23f97b-5764-4e14-9988-f43f784c1f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:20.557495 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23f97b-5764-4e14-9988-f43f784c1f2f admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:20.558331 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d23f97b-5764-4e14-9988-f43f784c1f2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:26:20.558646 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d23f97b-5764-4e14-9988-f43f784c1f2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:20.562458 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d23f97b-5764-4e14-9988-f43f784c1f2f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:20.603203 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 841/1687] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:26:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:26:20.604776 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d23f97b-5764-4e14-9988-f43f784c1f2f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:20.634423 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d23f97b-5764-4e14-9988-f43f784c1f2f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:20.647998 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 847/1688] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:20 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:20.654169 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fdf95d-6304-4966-b1e3-197e7250362b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:20.658859 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-84fdf95d-6304-4966-b1e3-197e7250362b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:20.664340 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b034ce5-8407-4647-a3ae-5988f074da48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:20.665016 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b034ce5-8407-4647-a3ae-5988f074da48 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:20.665267 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b034ce5-8407-4647-a3ae-5988f074da48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:20.665539 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b034ce5-8407-4647-a3ae-5988f074da48 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:20.666225 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b034ce5-8407-4647-a3ae-5988f074da48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:20.668200 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b034ce5-8407-4647-a3ae-5988f074da48 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:20.716679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fdf95d-6304-4966-b1e3-197e7250362b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:20.737958 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d6a7d9-58c4-4107-a8bf-e5e0ad64881d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:20.741247 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6a7d9-58c4-4107-a8bf-e5e0ad64881d admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:20.742084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6a7d9-58c4-4107-a8bf-e5e0ad64881d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:20.743222 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6a7d9-58c4-4107-a8bf-e5e0ad64881d admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:20.745234 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d6a7d9-58c4-4107-a8bf-e5e0ad64881d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:20.749988 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d6a7d9-58c4-4107-a8bf-e5e0ad64881d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:20.750813 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 842/1689] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:26:20 2022] POST /identity/v3/projects => generated 373 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:26:20.755109 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b034ce5-8407-4647-a3ae-5988f074da48 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:20.765211 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:20.866578 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 848/1690] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:26:20 2022] POST /identity/v3/projects => generated 403 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:26:20.869671 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d6a7d9-58c4-4107-a8bf-e5e0ad64881d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:20.955068 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b034ce5-8407-4647-a3ae-5988f074da48 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:21.036431 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d18c7b-8028-4eb0-9ab3-86f5acc5e2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:21.037776 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d18c7b-8028-4eb0-9ab3-86f5acc5e2fb admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:21.038121 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d18c7b-8028-4eb0-9ab3-86f5acc5e2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:21.038427 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d18c7b-8028-4eb0-9ab3-86f5acc5e2fb admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:21.039282 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d18c7b-8028-4eb0-9ab3-86f5acc5e2fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:21.041652 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d18c7b-8028-4eb0-9ab3-86f5acc5e2fb admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:21.082062 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 843/1691] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:26:20 2022] POST /identity/v3/users => generated 287 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:26:21.086394 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d18c7b-8028-4eb0-9ab3-86f5acc5e2fb admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:21.099210 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d18c7b-8028-4eb0-9ab3-86f5acc5e2fb admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:21.114299 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4464ac0-ce61-4d45-90f5-c45683618268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:21.115492 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4464ac0-ce61-4d45-90f5-c45683618268 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:21.115840 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4464ac0-ce61-4d45-90f5-c45683618268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:21.116179 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4464ac0-ce61-4d45-90f5-c45683618268 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:21.117827 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4464ac0-ce61-4d45-90f5-c45683618268 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:21.119753 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4464ac0-ce61-4d45-90f5-c45683618268 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:21.125266 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d6a7d9-58c4-4107-a8bf-e5e0ad64881d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:21.136401 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 844/1692] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:21 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:21.140126 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4464ac0-ce61-4d45-90f5-c45683618268 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:21.153795 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4629247d-2c54-487a-876c-224416b74571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:21.154465 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629247d-2c54-487a-876c-224416b74571 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:21.154751 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629247d-2c54-487a-876c-224416b74571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:21.155150 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629247d-2c54-487a-876c-224416b74571 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:21.155871 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4629247d-2c54-487a-876c-224416b74571 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:21.159108 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4629247d-2c54-487a-876c-224416b74571 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:21.173858 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4464ac0-ce61-4d45-90f5-c45683618268 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:21.183761 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ba138-10c0-45c8-a459-baa93b48bb4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:21.184425 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ba138-10c0-45c8-a459-baa93b48bb4b admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:21.193088 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ba138-10c0-45c8-a459-baa93b48bb4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:21.193534 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ba138-10c0-45c8-a459-baa93b48bb4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:21.194325 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ba138-10c0-45c8-a459-baa93b48bb4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:21.196176 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ba138-10c0-45c8-a459-baa93b48bb4b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:21.216225 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 845/1693] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:21 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:21.222731 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ba138-10c0-45c8-a459-baa93b48bb4b admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:21.234868 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 849/1694] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:26:20 2022] POST /identity/v3/users => generated 308 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:26:21.239054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4629247d-2c54-487a-876c-224416b74571 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:21.253514 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ba138-10c0-45c8-a459-baa93b48bb4b admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:21.258561 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4629247d-2c54-487a-876c-224416b74571 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:21.263613 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adce5be-88f2-4b05-a6ac-509599d6266b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:21.264402 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adce5be-88f2-4b05-a6ac-509599d6266b admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:21.264772 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adce5be-88f2-4b05-a6ac-509599d6266b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:21.265110 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adce5be-88f2-4b05-a6ac-509599d6266b admin admin] PATH_INFO: `/v3/projects/4f960ffa0fa3408fac7efa063d56054d/users/869e4ed766e74503a54fe3197fb641b6/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:21.292181 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ae056-ef0a-4958-a863-9d4e470f9dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:21.293395 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ae056-ef0a-4958-a863-9d4e470f9dab admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:21.293817 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ae056-ef0a-4958-a863-9d4e470f9dab admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:21.294238 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ae056-ef0a-4958-a863-9d4e470f9dab admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:21.295251 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8ae056-ef0a-4958-a863-9d4e470f9dab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:21.297065 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8ae056-ef0a-4958-a863-9d4e470f9dab admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:21.328445 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adce5be-88f2-4b05-a6ac-509599d6266b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f960ffa0fa3408fac7efa063d56054d, user_id=869e4ed766e74503a54fe3197fb641b6, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:21.339573 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 850/1695] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:21 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:21.340705 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ae056-ef0a-4958-a863-9d4e470f9dab admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:21.342733 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adce5be-88f2-4b05-a6ac-509599d6266b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:21.361666 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ae056-ef0a-4958-a863-9d4e470f9dab admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:21.373278 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 846/1696] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:26:21 2022] PUT /identity/v3/projects/4f960ffa0fa3408fac7efa063d56054d/users/869e4ed766e74503a54fe3197fb641b6/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:21.386592 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d81ae76-d810-4087-aedf-f7057689f39e None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:21.387079 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d81ae76-d810-4087-aedf-f7057689f39e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:21.387643 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d81ae76-d810-4087-aedf-f7057689f39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:21.447257 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-3d81ae76-d810-4087-aedf-f7057689f39e None None] MFA Rules not processed for user `869e4ed766e74503a54fe3197fb641b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:21.453115 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3d81ae76-d810-4087-aedf-f7057689f39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:21.456767 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:21.480702 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c16936-c63e-4719-b243-c5f3d4e8c27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:21.481693 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c16936-c63e-4719-b243-c5f3d4e8c27f admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:21.482072 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c16936-c63e-4719-b243-c5f3d4e8c27f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:21.482252 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c16936-c63e-4719-b243-c5f3d4e8c27f admin admin] PATH_INFO: `/v3/projects/c0ed0d86ce8347ee9ee421ce0730af5c/users/24386bd3e3c9484f98a57ad1546a73ce/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:21.504873 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c16936-c63e-4719-b243-c5f3d4e8c27f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0ed0d86ce8347ee9ee421ce0730af5c, user_id=24386bd3e3c9484f98a57ad1546a73ce, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:21.508954 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c16936-c63e-4719-b243-c5f3d4e8c27f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:21.531307 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 851/1697] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:26:21 2022] PUT /identity/v3/projects/c0ed0d86ce8347ee9ee421ce0730af5c/users/24386bd3e3c9484f98a57ad1546a73ce/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:21.538200 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c16936-c63e-4719-b243-c5f3d4e8c27f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:21.557703 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:21.614401 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 847/1698] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:21.620754 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d81ae76-d810-4087-aedf-f7057689f39e None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:21.640955 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d81ae76-d810-4087-aedf-f7057689f39e None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:21.654927 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c16936-c63e-4719-b243-c5f3d4e8c27f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:21.660709 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041280a-41d9-4854-b845-082aaad9ec27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:21.662215 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041280a-41d9-4854-b845-082aaad9ec27 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:21.662508 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041280a-41d9-4854-b845-082aaad9ec27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:21.662776 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041280a-41d9-4854-b845-082aaad9ec27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:21.663653 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4041280a-41d9-4854-b845-082aaad9ec27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:21.665337 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4041280a-41d9-4854-b845-082aaad9ec27 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:21.667489 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0fae1-f466-4d17-ae87-9ca491421dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:21.669980 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0fae1-f466-4d17-ae87-9ca491421dbe admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:21.674421 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0fae1-f466-4d17-ae87-9ca491421dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:21.674868 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0fae1-f466-4d17-ae87-9ca491421dbe admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:21.675795 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c0fae1-f466-4d17-ae87-9ca491421dbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:21.678307 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c0fae1-f466-4d17-ae87-9ca491421dbe admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:21.683174 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 852/1699] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:21 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:21.684298 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041280a-41d9-4854-b845-082aaad9ec27 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:21.698710 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041280a-41d9-4854-b845-082aaad9ec27 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:21.704841 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8357aab6-749d-4216-8870-905bb7bee719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:21.705854 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357aab6-749d-4216-8870-905bb7bee719 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:21.706137 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357aab6-749d-4216-8870-905bb7bee719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:21.706407 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357aab6-749d-4216-8870-905bb7bee719 admin admin] PATH_INFO: `/v3/projects/c0ed0d86ce8347ee9ee421ce0730af5c/users/24386bd3e3c9484f98a57ad1546a73ce/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:21.709249 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8357aab6-749d-4216-8870-905bb7bee719 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0ed0d86ce8347ee9ee421ce0730af5c, user_id=24386bd3e3c9484f98a57ad1546a73ce, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:21.718256 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8357aab6-749d-4216-8870-905bb7bee719 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:21.738179 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 853/1700] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:26:21 2022] PUT /identity/v3/projects/c0ed0d86ce8347ee9ee421ce0730af5c/users/24386bd3e3c9484f98a57ad1546a73ce/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:21.748446 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 848/1701] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:26:21 2022] POST /identity/v3/projects => generated 373 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:26:21.749749 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8357aab6-749d-4216-8870-905bb7bee719 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:21.835918 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515260d-861a-45bd-a0dc-867406d1d23c None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:21.836335 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515260d-861a-45bd-a0dc-867406d1d23c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:21.836733 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515260d-861a-45bd-a0dc-867406d1d23c None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:21.839821 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 849/1702] 10.222.0.40 () {54 vars in 912 bytes} [Fri Aug 5 10:26:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 10:26:21.853100 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5445a129-8f42-4c60-bdee-5ae85b54ce03 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:21.853921 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5445a129-8f42-4c60-bdee-5ae85b54ce03 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:21.854218 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5445a129-8f42-4c60-bdee-5ae85b54ce03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:21.863145 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8357aab6-749d-4216-8870-905bb7bee719 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:21.933591 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-5445a129-8f42-4c60-bdee-5ae85b54ce03 None None] MFA Rules not processed for user `a7deb94069a449269cb03406232174d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:21.937612 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5445a129-8f42-4c60-bdee-5ae85b54ce03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:22.087783 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 850/1703] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 5 10:26:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:26:22.098782 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6434bea-472d-4fbc-a318-5b01bd3edfee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:22.106712 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6434bea-472d-4fbc-a318-5b01bd3edfee admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:22.108518 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6434bea-472d-4fbc-a318-5b01bd3edfee admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:22.108814 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6434bea-472d-4fbc-a318-5b01bd3edfee admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:22.109529 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6434bea-472d-4fbc-a318-5b01bd3edfee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:22.122657 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6434bea-472d-4fbc-a318-5b01bd3edfee admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:22.154390 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5445a129-8f42-4c60-bdee-5ae85b54ce03 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:22.157206 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5445a129-8f42-4c60-bdee-5ae85b54ce03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:22.163044 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:22.185806 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5445a129-8f42-4c60-bdee-5ae85b54ce03 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:22.197297 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 854/1704] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:26:21 2022] POST /identity/v3/users => generated 293 bytes in 447 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:26:22.200794 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6434bea-472d-4fbc-a318-5b01bd3edfee admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:22.202405 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec87e0c8-fc7c-412e-ac32-bd2a2e938852 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:22.203086 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87e0c8-fc7c-412e-ac32-bd2a2e938852 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:22.203371 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87e0c8-fc7c-412e-ac32-bd2a2e938852 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:22.203652 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87e0c8-fc7c-412e-ac32-bd2a2e938852 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:22.232651 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec87e0c8-fc7c-412e-ac32-bd2a2e938852 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:22.234519 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec87e0c8-fc7c-412e-ac32-bd2a2e938852 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:22.237364 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6434bea-472d-4fbc-a318-5b01bd3edfee admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:22.257670 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1af81c-9dd6-4b0c-88fd-b67e3baaca0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:22.258640 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1af81c-9dd6-4b0c-88fd-b67e3baaca0f admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:22.258945 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1af81c-9dd6-4b0c-88fd-b67e3baaca0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:22.263039 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1af81c-9dd6-4b0c-88fd-b67e3baaca0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:22.266084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1af81c-9dd6-4b0c-88fd-b67e3baaca0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:22.271367 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 851/1705] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:26:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:26:22.275650 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1af81c-9dd6-4b0c-88fd-b67e3baaca0f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:22.311825 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec87e0c8-fc7c-412e-ac32-bd2a2e938852 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:22.323295 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 855/1706] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:22.329679 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec87e0c8-fc7c-412e-ac32-bd2a2e938852 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:22.348737 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba029613-3ca0-46da-8b39-7798bc1a4a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:22.349734 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba029613-3ca0-46da-8b39-7798bc1a4a15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:22.350872 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba029613-3ca0-46da-8b39-7798bc1a4a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:22.351226 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba029613-3ca0-46da-8b39-7798bc1a4a15 admin admin] PATH_INFO: `/v3/projects/2dfdab2067934acdb274b9314bfdb3f7/users/e6f9b803b9ef4b4b97abf62ab0369514/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:22.391107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba029613-3ca0-46da-8b39-7798bc1a4a15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dfdab2067934acdb274b9314bfdb3f7, user_id=e6f9b803b9ef4b4b97abf62ab0369514, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:22.395041 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba029613-3ca0-46da-8b39-7798bc1a4a15 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:22.429886 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1af81c-9dd6-4b0c-88fd-b67e3baaca0f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:22.430358 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 852/1707] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:26:22 2022] PUT /identity/v3/projects/2dfdab2067934acdb274b9314bfdb3f7/users/e6f9b803b9ef4b4b97abf62ab0369514/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:22.442092 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1af81c-9dd6-4b0c-88fd-b67e3baaca0f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:22.476959 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba029613-3ca0-46da-8b39-7798bc1a4a15 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:22.478700 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ba029613-3ca0-46da-8b39-7798bc1a4a15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:22.682113 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2846e808-622b-422e-9110-c5da82bf8187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:22.683225 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2846e808-622b-422e-9110-c5da82bf8187 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:22.683512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2846e808-622b-422e-9110-c5da82bf8187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:22.683793 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2846e808-622b-422e-9110-c5da82bf8187 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:22.684532 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2846e808-622b-422e-9110-c5da82bf8187 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:22.686222 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2846e808-622b-422e-9110-c5da82bf8187 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:22.715644 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 856/1708] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:22.717056 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2846e808-622b-422e-9110-c5da82bf8187 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:22.723459 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba029613-3ca0-46da-8b39-7798bc1a4a15 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:22.735417 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbf7d50-631a-48b0-8ae0-1dec6f5a41d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:22.736343 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf7d50-631a-48b0-8ae0-1dec6f5a41d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:22.736726 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf7d50-631a-48b0-8ae0-1dec6f5a41d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:22.737099 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf7d50-631a-48b0-8ae0-1dec6f5a41d8 admin admin] PATH_INFO: `/v3/users/318d06f754a4443eb20a3184a3d27d61` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:22.737730 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2846e808-622b-422e-9110-c5da82bf8187 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:22.738581 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbf7d50-631a-48b0-8ae0-1dec6f5a41d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=318d06f754a4443eb20a3184a3d27d61)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:22.740667 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbf7d50-631a-48b0-8ae0-1dec6f5a41d8 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:22.753525 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b7373-c812-448e-a63a-931f2150c834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:22.755772 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b7373-c812-448e-a63a-931f2150c834 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:22.756602 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b7373-c812-448e-a63a-931f2150c834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:22.756602 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b7373-c812-448e-a63a-931f2150c834 admin admin] PATH_INFO: `/v3/projects/2dfdab2067934acdb274b9314bfdb3f7/users/e6f9b803b9ef4b4b97abf62ab0369514/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:22.764033 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308b7373-c812-448e-a63a-931f2150c834 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dfdab2067934acdb274b9314bfdb3f7, user_id=e6f9b803b9ef4b4b97abf62ab0369514, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:22.772828 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308b7373-c812-448e-a63a-931f2150c834 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:22.793427 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 857/1709] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:26:22 2022] PUT /identity/v3/projects/2dfdab2067934acdb274b9314bfdb3f7/users/e6f9b803b9ef4b4b97abf62ab0369514/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:22.804766 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:22.809119 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c0a60-5854-4b38-bbbb-cd6ee0e127a0 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:22.809441 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c0a60-5854-4b38-bbbb-cd6ee0e127a0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:22.809751 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c0a60-5854-4b38-bbbb-cd6ee0e127a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:22.868734 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6fbf7d50-631a-48b0-8ae0-1dec6f5a41d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '318d06f754a4443eb20a3184a3d27d61'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:22.884570 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6fbf7d50-631a-48b0-8ae0-1dec6f5a41d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '318d06f754a4443eb20a3184a3d27d61'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:22.885412 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6fbf7d50-631a-48b0-8ae0-1dec6f5a41d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '318d06f754a4443eb20a3184a3d27d61'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:22.887824 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1a5c0a60-5854-4b38-bbbb-cd6ee0e127a0 None None] MFA Rules not processed for user `e6f9b803b9ef4b4b97abf62ab0369514`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:22.889306 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1a5c0a60-5854-4b38-bbbb-cd6ee0e127a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:22.912712 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6fbf7d50-631a-48b0-8ae0-1dec6f5a41d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '318d06f754a4443eb20a3184a3d27d61'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:22.916268 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6fbf7d50-631a-48b0-8ae0-1dec6f5a41d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '318d06f754a4443eb20a3184a3d27d61'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:22.961802 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 853/1710] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:26:22 2022] DELETE /identity/v3/users/318d06f754a4443eb20a3184a3d27d61 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:22.963644 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fecd70-f0cd-48a5-9919-f85442b004d3 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:22.964035 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fecd70-f0cd-48a5-9919-f85442b004d3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:22.964415 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fecd70-f0cd-48a5-9919-f85442b004d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:23.012418 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-d7fecd70-f0cd-48a5-9919-f85442b004d3 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:23.014733 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d7fecd70-f0cd-48a5-9919-f85442b004d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:23.127905 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 858/1711] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:22 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:23.164990 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5c0a60-5854-4b38-bbbb-cd6ee0e127a0 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:23.167088 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1a5c0a60-5854-4b38-bbbb-cd6ee0e127a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:23.188544 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 854/1712] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:23.189961 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f495b5-682a-4861-850d-bdb25c9b6a18 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:23.190249 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f495b5-682a-4861-850d-bdb25c9b6a18 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:23.190522 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f495b5-682a-4861-850d-bdb25c9b6a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:23.231026 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-07f495b5-682a-4861-850d-bdb25c9b6a18 None None] MFA Rules not processed for user `9ace48889c484e4fb5fed0f8f80c3288`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:23.233195 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-07f495b5-682a-4861-850d-bdb25c9b6a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:23.300550 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:23.314363 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5c0a60-5854-4b38-bbbb-cd6ee0e127a0 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:23.420505 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 855/1713] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:23.422965 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f495b5-682a-4861-850d-bdb25c9b6a18 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:23.425080 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-07f495b5-682a-4861-850d-bdb25c9b6a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:23.465175 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1371c1ad-81b6-4008-b3db-64d72a97fa0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:23.465818 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1371c1ad-81b6-4008-b3db-64d72a97fa0b service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:23.466131 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1371c1ad-81b6-4008-b3db-64d72a97fa0b service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:23.466436 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1371c1ad-81b6-4008-b3db-64d72a97fa0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:23.492420 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:23.560517 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f495b5-682a-4861-850d-bdb25c9b6a18 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:23.571680 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be09c920-7ae7-4f07-a372-acc379a6bb24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:23.573439 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09c920-7ae7-4f07-a372-acc379a6bb24 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:23.576019 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09c920-7ae7-4f07-a372-acc379a6bb24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:23.576433 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09c920-7ae7-4f07-a372-acc379a6bb24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:23.602513 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be09c920-7ae7-4f07-a372-acc379a6bb24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:23.605137 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be09c920-7ae7-4f07-a372-acc379a6bb24 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:23.643229 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1371c1ad-81b6-4008-b3db-64d72a97fa0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:23.644711 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1371c1ad-81b6-4008-b3db-64d72a97fa0b service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:23.652851 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 856/1714] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:26:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:26:23.656626 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd3f44-6524-414b-be49-ffad65258b12 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:23.656974 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd3f44-6524-414b-be49-ffad65258b12 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:23.657185 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd3f44-6524-414b-be49-ffad65258b12 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:23.659184 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 857/1715] 10.222.0.40 () {54 vars in 912 bytes} [Fri Aug 5 10:26:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:26:23.659987 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44d70f-5c7f-4fd5-aeb0-60a438e8212a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:23.660271 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44d70f-5c7f-4fd5-aeb0-60a438e8212a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:23.660547 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44d70f-5c7f-4fd5-aeb0-60a438e8212a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:23.679928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 859/1716] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:26:23 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:26:23.687626 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b6f24-3cef-4263-9388-251070e7678a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:23.688263 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b6f24-3cef-4263-9388-251070e7678a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:23.688732 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b6f24-3cef-4263-9388-251070e7678a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:23.737980 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-0f44d70f-5c7f-4fd5-aeb0-60a438e8212a None None] MFA Rules not processed for user `9ace48889c484e4fb5fed0f8f80c3288`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:23.740535 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0f44d70f-5c7f-4fd5-aeb0-60a438e8212a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:23.764173 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 858/1717] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:23.768141 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf255e15-4a0a-4baf-a9f7-49b7a45afdfd None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:23.768435 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf255e15-4a0a-4baf-a9f7-49b7a45afdfd None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:23.768717 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf255e15-4a0a-4baf-a9f7-49b7a45afdfd None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:23.770098 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 859/1718] 10.222.0.40 () {54 vars in 961 bytes} [Fri Aug 5 10:26:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:26:23.773099 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-5d2b6f24-3cef-4263-9388-251070e7678a None None] MFA Rules not processed for user `a7deb94069a449269cb03406232174d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:23.774575 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5d2b6f24-3cef-4263-9388-251070e7678a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:23.776911 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1492de4a-91d0-4554-8fdb-8d648d74a241 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:23.777200 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1492de4a-91d0-4554-8fdb-8d648d74a241 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:23.777480 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1492de4a-91d0-4554-8fdb-8d648d74a241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:23.794068 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 860/1719] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 5 10:26:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:23.866455 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-1492de4a-91d0-4554-8fdb-8d648d74a241 None None] MFA Rules not processed for user `8743f5bb0be54d57a957617636b0b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:23.867969 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1492de4a-91d0-4554-8fdb-8d648d74a241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:23.933975 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2b6f24-3cef-4263-9388-251070e7678a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:23.933975 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5d2b6f24-3cef-4263-9388-251070e7678a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:24.098529 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 860/1720] 10.222.0.40 () {58 vars in 1058 bytes} [Fri Aug 5 10:26:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:26:24.110362 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c168878-e798-4ce8-8f35-c5637dd0257b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:24.111406 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c168878-e798-4ce8-8f35-c5637dd0257b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:24.117644 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c168878-e798-4ce8-8f35-c5637dd0257b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:24.169925 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-5c168878-e798-4ce8-8f35-c5637dd0257b None None] MFA Rules not processed for user `a3b4d90bcfc34d2d9ba1c40316e09858`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:24.171363 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5c168878-e798-4ce8-8f35-c5637dd0257b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:24.254948 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2b6f24-3cef-4263-9388-251070e7678a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:24.267881 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 861/1721] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:24 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:24.268954 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c168878-e798-4ce8-8f35-c5637dd0257b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:24.270763 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5c168878-e798-4ce8-8f35-c5637dd0257b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:24.272725 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb68aa-3422-4595-8b44-9e9d172f52f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:24.273305 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb68aa-3422-4595-8b44-9e9d172f52f8 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:24.273583 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb68aa-3422-4595-8b44-9e9d172f52f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:24.273855 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb68aa-3422-4595-8b44-9e9d172f52f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:24.288284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c168878-e798-4ce8-8f35-c5637dd0257b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:24.297800 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b4d49f-426b-4638-b50e-b0fa01302fb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:24.298727 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b4d49f-426b-4638-b50e-b0fa01302fb2 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:24.299005 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b4d49f-426b-4638-b50e-b0fa01302fb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:24.299338 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b4d49f-426b-4638-b50e-b0fa01302fb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:24.305060 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb68aa-3422-4595-8b44-9e9d172f52f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:24.307187 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb68aa-3422-4595-8b44-9e9d172f52f8 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:24.311088 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b4d49f-426b-4638-b50e-b0fa01302fb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:24.312687 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b4d49f-426b-4638-b50e-b0fa01302fb2 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:24.323046 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 862/1722] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:26:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:26:24.324007 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b4d49f-426b-4638-b50e-b0fa01302fb2 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:24.334683 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b4d49f-426b-4638-b50e-b0fa01302fb2 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:24.350569 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b646d8-0151-4568-99d1-1b94b02ea4f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:24.350718 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 861/1723] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:26:23 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 10:26:24.351489 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb68aa-3422-4595-8b44-9e9d172f52f8 service nova] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:24.351697 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b646d8-0151-4568-99d1-1b94b02ea4f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:24.352083 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b646d8-0151-4568-99d1-1b94b02ea4f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:24.352546 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b646d8-0151-4568-99d1-1b94b02ea4f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:24.356955 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-33b646d8-0151-4568-99d1-1b94b02ea4f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:24.395848 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb68aa-3422-4595-8b44-9e9d172f52f8 service nova] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:24.418474 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2439be-b3b5-4724-bacf-230835571cb4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8743f5bb0be54d57a957617636b0b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:24.419154 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2439be-b3b5-4724-bacf-230835571cb4 service placement] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:24.419657 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2439be-b3b5-4724-bacf-230835571cb4 service placement] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:24.420067 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2439be-b3b5-4724-bacf-230835571cb4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:24.428631 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-af2439be-b3b5-4724-bacf-230835571cb4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:24.463570 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2439be-b3b5-4724-bacf-230835571cb4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:24.465223 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2439be-b3b5-4724-bacf-230835571cb4 service placement] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:24.496614 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:24.515603 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 862/1724] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:26:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:26:24.520320 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b646d8-0151-4568-99d1-1b94b02ea4f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:24.521934 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b646d8-0151-4568-99d1-1b94b02ea4f9 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:24.533462 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 863/1725] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:26:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:26:24.875463 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2439be-b3b5-4724-bacf-230835571cb4 service placement] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:24.877623 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-af2439be-b3b5-4724-bacf-230835571cb4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:24.918076 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2439be-b3b5-4724-bacf-230835571cb4 service placement] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:24.933014 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1150f9-8e55-465c-9828-e0fc4f0c47e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:24.935453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1150f9-8e55-465c-9828-e0fc4f0c47e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:24.936690 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1150f9-8e55-465c-9828-e0fc4f0c47e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:24.938780 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1150f9-8e55-465c-9828-e0fc4f0c47e4 admin admin] PATH_INFO: `/v3/projects/3e57f4b8241c4650940c57d675e39fab` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:24.943096 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1150f9-8e55-465c-9828-e0fc4f0c47e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e57f4b8241c4650940c57d675e39fab)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:24.955877 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1150f9-8e55-465c-9828-e0fc4f0c47e4 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:25.014621 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:25.174812 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7d1150f9-8e55-465c-9828-e0fc4f0c47e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e57f4b8241c4650940c57d675e39fab', 'request_id': 'req-7d1150f9-8e55-465c-9828-e0fc4f0c47e4'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:25.203795 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7d1150f9-8e55-465c-9828-e0fc4f0c47e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e57f4b8241c4650940c57d675e39fab', 'request_id': 'req-7d1150f9-8e55-465c-9828-e0fc4f0c47e4'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:25.216101 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7d1150f9-8e55-465c-9828-e0fc4f0c47e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e57f4b8241c4650940c57d675e39fab', 'request_id': 'req-7d1150f9-8e55-465c-9828-e0fc4f0c47e4'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:25.248527 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7d1150f9-8e55-465c-9828-e0fc4f0c47e4 admin admin] The token cache is being invalidate because project 3e57f4b8241c4650940c57d675e39fab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:26:25.249064 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7d1150f9-8e55-465c-9828-e0fc4f0c47e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:25.252721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7d1150f9-8e55-465c-9828-e0fc4f0c47e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:25.255247 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 863/1726] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:26:24 2022] DELETE /identity/v3/projects/3e57f4b8241c4650940c57d675e39fab => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:25.259677 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b646d8-0151-4568-99d1-1b94b02ea4f9 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:25.293180 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b646d8-0151-4568-99d1-1b94b02ea4f9 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:25.422510 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db88424c-3f7d-4994-8c62-792480699f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:25.424968 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db88424c-3f7d-4994-8c62-792480699f81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:25.425700 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db88424c-3f7d-4994-8c62-792480699f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:25.426256 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db88424c-3f7d-4994-8c62-792480699f81 admin admin] PATH_INFO: `/v3/users/e74abcfb8cde47d18ecec1ded868d2ed` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:25.428628 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db88424c-3f7d-4994-8c62-792480699f81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e74abcfb8cde47d18ecec1ded868d2ed)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:25.432113 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db88424c-3f7d-4994-8c62-792480699f81 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:25.641286 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-db88424c-3f7d-4994-8c62-792480699f81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e74abcfb8cde47d18ecec1ded868d2ed'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:25.665364 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-db88424c-3f7d-4994-8c62-792480699f81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e74abcfb8cde47d18ecec1ded868d2ed'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:25.666378 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-db88424c-3f7d-4994-8c62-792480699f81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e74abcfb8cde47d18ecec1ded868d2ed'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:25.694303 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-db88424c-3f7d-4994-8c62-792480699f81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e74abcfb8cde47d18ecec1ded868d2ed'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:25.695918 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-db88424c-3f7d-4994-8c62-792480699f81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e74abcfb8cde47d18ecec1ded868d2ed'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:25.732055 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 864/1727] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:26:25 2022] DELETE /identity/v3/users/e74abcfb8cde47d18ecec1ded868d2ed => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:26:27.019571 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a804454f-30eb-46b5-8dbb-d19e7775f287 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:27.020212 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a804454f-30eb-46b5-8dbb-d19e7775f287 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:27.020212 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a804454f-30eb-46b5-8dbb-d19e7775f287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:27.092090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a804454f-30eb-46b5-8dbb-d19e7775f287 None None] MFA Rules not processed for user `24386bd3e3c9484f98a57ad1546a73ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:27.095233 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a804454f-30eb-46b5-8dbb-d19e7775f287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:27.237517 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 864/1728] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:27 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:28.535479 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d62632-9f09-45c0-bc2a-98802014d892 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:28.536088 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d62632-9f09-45c0-bc2a-98802014d892 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:28.536201 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d62632-9f09-45c0-bc2a-98802014d892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:28.690752 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-79d62632-9f09-45c0-bc2a-98802014d892 None None] MFA Rules not processed for user `52fc3ba90dc045c2a5b7bb730da96acc`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:28.692255 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-79d62632-9f09-45c0-bc2a-98802014d892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:28.863188 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:28.892702 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a804454f-30eb-46b5-8dbb-d19e7775f287 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:28.897666 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a804454f-30eb-46b5-8dbb-d19e7775f287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:28.949286 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 865/1729] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 5 10:26:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 415 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:28.953086 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e3e732-48ab-483b-aa8b-ac52dbb8e1a0 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:28.958462 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e3e732-48ab-483b-aa8b-ac52dbb8e1a0 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:28.958841 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e3e732-48ab-483b-aa8b-ac52dbb8e1a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:29.058723 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c4e3e732-48ab-483b-aa8b-ac52dbb8e1a0 None None] MFA Rules not processed for user `7e7eb355fac04252a4be1f8e60037924`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:29.060277 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c4e3e732-48ab-483b-aa8b-ac52dbb8e1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:29.079146 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a804454f-30eb-46b5-8dbb-d19e7775f287 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:29.097393 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c4a27-df4d-4c78-89d2-4a7ede4b7c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:29.098580 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c4a27-df4d-4c78-89d2-4a7ede4b7c37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:29.099331 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c4a27-df4d-4c78-89d2-4a7ede4b7c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:29.099331 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c4a27-df4d-4c78-89d2-4a7ede4b7c37 admin admin] PATH_INFO: `/v3/projects/bd69bbc96ce349f3a9cd44669c50c9e3` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:29.101586 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c4a27-df4d-4c78-89d2-4a7ede4b7c37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd69bbc96ce349f3a9cd44669c50c9e3)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:29.103821 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c4a27-df4d-4c78-89d2-4a7ede4b7c37 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:29.183337 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:29.271866 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-714c4a27-df4d-4c78-89d2-4a7ede4b7c37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd69bbc96ce349f3a9cd44669c50c9e3', 'request_id': 'req-714c4a27-df4d-4c78-89d2-4a7ede4b7c37'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:29.304708 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-714c4a27-df4d-4c78-89d2-4a7ede4b7c37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd69bbc96ce349f3a9cd44669c50c9e3', 'request_id': 'req-714c4a27-df4d-4c78-89d2-4a7ede4b7c37'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:29.310755 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 866/1730] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:29.313061 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e3e732-48ab-483b-aa8b-ac52dbb8e1a0 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:29.318130 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c4e3e732-48ab-483b-aa8b-ac52dbb8e1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:29.339331 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-714c4a27-df4d-4c78-89d2-4a7ede4b7c37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd69bbc96ce349f3a9cd44669c50c9e3', 'request_id': 'req-714c4a27-df4d-4c78-89d2-4a7ede4b7c37'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:29.365697 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-714c4a27-df4d-4c78-89d2-4a7ede4b7c37 admin admin] The token cache is being invalidate because project bd69bbc96ce349f3a9cd44669c50c9e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:26:29.366263 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-714c4a27-df4d-4c78-89d2-4a7ede4b7c37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:29.368310 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-714c4a27-df4d-4c78-89d2-4a7ede4b7c37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:29.374123 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 865/1731] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:26:28 2022] DELETE /identity/v3/projects/bd69bbc96ce349f3a9cd44669c50c9e3 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:26:29.375489 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c4a27-df4d-4c78-89d2-4a7ede4b7c37 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:29.378891 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-714c4a27-df4d-4c78-89d2-4a7ede4b7c37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:29.552261 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e3e732-48ab-483b-aa8b-ac52dbb8e1a0 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:29.659929 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c4a27-df4d-4c78-89d2-4a7ede4b7c37 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:29.680315 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcabe99-48de-40d0-9ecc-2521dd4d53d3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c80cf0e429f04f229aea1aa9293a93b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:29.680987 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcabe99-48de-40d0-9ecc-2521dd4d53d3 service swift] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:29.681276 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcabe99-48de-40d0-9ecc-2521dd4d53d3 service swift] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:29.681554 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcabe99-48de-40d0-9ecc-2521dd4d53d3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:29.686169 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-dfcabe99-48de-40d0-9ecc-2521dd4d53d3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:29.732542 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb2f3f-0a08-4d5e-a4aa-6a810868986f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:29.733355 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb2f3f-0a08-4d5e-a4aa-6a810868986f service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:29.733740 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb2f3f-0a08-4d5e-a4aa-6a810868986f service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:29.734196 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb2f3f-0a08-4d5e-a4aa-6a810868986f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:29.742196 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8eeb2f3f-0a08-4d5e-a4aa-6a810868986f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:29.754699 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:29.909052 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcabe99-48de-40d0-9ecc-2521dd4d53d3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:29.910854 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcabe99-48de-40d0-9ecc-2521dd4d53d3 service swift] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:29.932145 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeb2f3f-0a08-4d5e-a4aa-6a810868986f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:29.934684 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeb2f3f-0a08-4d5e-a4aa-6a810868986f service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:29.950862 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 867/1732] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:26:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 639 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:26:29.952608 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299dfd0e-1e47-4894-8240-645aaec27f30 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:29.953079 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299dfd0e-1e47-4894-8240-645aaec27f30 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:29.953308 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299dfd0e-1e47-4894-8240-645aaec27f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:29.965961 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 866/1733] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 5 10:26:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 591 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 10:26:30.000408 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-299dfd0e-1e47-4894-8240-645aaec27f30 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:30.002301 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-299dfd0e-1e47-4894-8240-645aaec27f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:30.250924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 868/1734] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:30.254942 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a7470-abc2-48f8-9438-30b7f1d55e7d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:30.255146 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a7470-abc2-48f8-9438-30b7f1d55e7d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:30.255434 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a7470-abc2-48f8-9438-30b7f1d55e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:30.303334 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-3e9a7470-abc2-48f8-9438-30b7f1d55e7d None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:30.306195 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3e9a7470-abc2-48f8-9438-30b7f1d55e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:30.329461 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 867/1735] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:30.332565 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299dfd0e-1e47-4894-8240-645aaec27f30 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:30.345860 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299dfd0e-1e47-4894-8240-645aaec27f30 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:30.370322 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8093f8-dc77-46ca-81c9-deffcf67bd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:30.371192 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8093f8-dc77-46ca-81c9-deffcf67bd39 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:30.371479 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8093f8-dc77-46ca-81c9-deffcf67bd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:30.371738 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8093f8-dc77-46ca-81c9-deffcf67bd39 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:30.372580 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8093f8-dc77-46ca-81c9-deffcf67bd39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:26:30.372759 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8093f8-dc77-46ca-81c9-deffcf67bd39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:30.375912 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8093f8-dc77-46ca-81c9-deffcf67bd39 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:30.392747 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 869/1736] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:26:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:26:30.397132 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9a7470-abc2-48f8-9438-30b7f1d55e7d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:30.414893 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9a7470-abc2-48f8-9438-30b7f1d55e7d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:30.420168 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36974e17-f37f-483e-a374-a2df994e247e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:30.421079 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36974e17-f37f-483e-a374-a2df994e247e admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:30.421563 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36974e17-f37f-483e-a374-a2df994e247e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:30.422068 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36974e17-f37f-483e-a374-a2df994e247e admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:30.423182 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36974e17-f37f-483e-a374-a2df994e247e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:30.427766 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36974e17-f37f-483e-a374-a2df994e247e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:30.477860 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 868/1737] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:26:30 2022] POST /identity/v3/projects => generated 375 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:26:30.482784 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8093f8-dc77-46ca-81c9-deffcf67bd39 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:30.497591 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8093f8-dc77-46ca-81c9-deffcf67bd39 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:30.692101 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7dd386-5ebe-4532-9a56-1f9fcd1116d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:30.692785 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7dd386-5ebe-4532-9a56-1f9fcd1116d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:30.693047 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7dd386-5ebe-4532-9a56-1f9fcd1116d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:30.693323 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7dd386-5ebe-4532-9a56-1f9fcd1116d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:30.694019 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7dd386-5ebe-4532-9a56-1f9fcd1116d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:30.696968 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7dd386-5ebe-4532-9a56-1f9fcd1116d8 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:30.723854 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6944cc6-759d-4e3e-940f-4a3b77fb2264 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:30.724354 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6944cc6-759d-4e3e-940f-4a3b77fb2264 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:30.724974 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6944cc6-759d-4e3e-940f-4a3b77fb2264 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:30.726567 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 869/1738] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 5 10:26:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:26:30.755443 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 870/1739] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:26:30 2022] POST /identity/v3/users => generated 288 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:26:30.756274 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6944cc6-759d-4e3e-940f-4a3b77fb2264 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:30.785989 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6944cc6-759d-4e3e-940f-4a3b77fb2264 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:30.802257 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d7ef69-a727-4b21-8e01-2fbc88cd169d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:30.804548 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7ef69-a727-4b21-8e01-2fbc88cd169d admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:30.804548 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7ef69-a727-4b21-8e01-2fbc88cd169d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:30.804548 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7ef69-a727-4b21-8e01-2fbc88cd169d admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:30.805585 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d7ef69-a727-4b21-8e01-2fbc88cd169d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:30.810343 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d7ef69-a727-4b21-8e01-2fbc88cd169d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:30.817819 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:30.836074 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 870/1740] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:30.840408 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7dd386-5ebe-4532-9a56-1f9fcd1116d8 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:30.856598 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7dd386-5ebe-4532-9a56-1f9fcd1116d8 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:30.873262 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b09438-702b-4e26-a0d0-8cd0b39d9adc None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:30.874020 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b09438-702b-4e26-a0d0-8cd0b39d9adc None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:30.874503 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b09438-702b-4e26-a0d0-8cd0b39d9adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:30.875079 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a65b9b0-0aae-4832-b6c4-0f4a921cb9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:30.876106 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a65b9b0-0aae-4832-b6c4-0f4a921cb9bf admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:30.876484 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a65b9b0-0aae-4832-b6c4-0f4a921cb9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:30.879444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a65b9b0-0aae-4832-b6c4-0f4a921cb9bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:30.880390 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a65b9b0-0aae-4832-b6c4-0f4a921cb9bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:30.884960 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a65b9b0-0aae-4832-b6c4-0f4a921cb9bf admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:30.920533 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d3b09438-702b-4e26-a0d0-8cd0b39d9adc None None] MFA Rules not processed for user `8743f5bb0be54d57a957617636b0b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:30.928208 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d3b09438-702b-4e26-a0d0-8cd0b39d9adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:30.940975 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 871/1741] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:30.942151 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a65b9b0-0aae-4832-b6c4-0f4a921cb9bf admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:31.008932 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a65b9b0-0aae-4832-b6c4-0f4a921cb9bf admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:31.032151 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d658aa29-4f9c-47ab-9bbd-b4624dc42411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:31.032915 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658aa29-4f9c-47ab-9bbd-b4624dc42411 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:31.033196 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658aa29-4f9c-47ab-9bbd-b4624dc42411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:31.033472 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658aa29-4f9c-47ab-9bbd-b4624dc42411 admin admin] PATH_INFO: `/v3/projects/ca2bf86444e54a5abf161b8161a015a4/users/8b221fdbe86f45b7ae929c1f9b4af448/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:31.061669 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:31.073493 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d658aa29-4f9c-47ab-9bbd-b4624dc42411 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca2bf86444e54a5abf161b8161a015a4, user_id=8b221fdbe86f45b7ae929c1f9b4af448, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:31.080607 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d658aa29-4f9c-47ab-9bbd-b4624dc42411 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:31.112737 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 872/1742] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:26:30 2022] PUT /identity/v3/projects/ca2bf86444e54a5abf161b8161a015a4/users/8b221fdbe86f45b7ae929c1f9b4af448/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:31.116677 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e454fe06-baf8-4139-b22d-24b2f9613aff None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:31.117016 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e454fe06-baf8-4139-b22d-24b2f9613aff None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:31.117292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e454fe06-baf8-4139-b22d-24b2f9613aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:31.140797 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 871/1743] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 5 10:26:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:31.166717 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-e454fe06-baf8-4139-b22d-24b2f9613aff None None] MFA Rules not processed for user `8b221fdbe86f45b7ae929c1f9b4af448`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:31.168285 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e454fe06-baf8-4139-b22d-24b2f9613aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:31.201219 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b09438-702b-4e26-a0d0-8cd0b39d9adc None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:31.202944 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d3b09438-702b-4e26-a0d0-8cd0b39d9adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:31.233282 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b09438-702b-4e26-a0d0-8cd0b39d9adc None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:31.282970 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 873/1744] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:26:31.287202 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e454fe06-baf8-4139-b22d-24b2f9613aff None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:31.401100 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50ba5fc-8509-461a-b2c0-073deb151cfa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8743f5bb0be54d57a957617636b0b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:31.402160 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ba5fc-8509-461a-b2c0-073deb151cfa service placement] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:31.402691 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ba5fc-8509-461a-b2c0-073deb151cfa service placement] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:31.403147 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ba5fc-8509-461a-b2c0-073deb151cfa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:31.431043 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50ba5fc-8509-461a-b2c0-073deb151cfa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:31.435658 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50ba5fc-8509-461a-b2c0-073deb151cfa service placement] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:31.436491 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e454fe06-baf8-4139-b22d-24b2f9613aff None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:31.442304 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1283d6-968c-4a44-9836-2a09c02aeed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:31.443062 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1283d6-968c-4a44-9836-2a09c02aeed7 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:31.443334 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1283d6-968c-4a44-9836-2a09c02aeed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:31.443608 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1283d6-968c-4a44-9836-2a09c02aeed7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:31.444604 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1283d6-968c-4a44-9836-2a09c02aeed7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:31.446753 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1283d6-968c-4a44-9836-2a09c02aeed7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:31.460851 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 872/1745] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:26:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 10:26:31.510722 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 874/1746] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:26:31 2022] POST /identity/v3/projects => generated 379 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:26:31.512126 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50ba5fc-8509-461a-b2c0-073deb151cfa service placement] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:31.526764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50ba5fc-8509-461a-b2c0-073deb151cfa service placement] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:31.661941 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110df4fd-deab-4880-a561-69ea0016203d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:31.662735 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110df4fd-deab-4880-a561-69ea0016203d admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:31.663167 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110df4fd-deab-4880-a561-69ea0016203d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:31.663540 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110df4fd-deab-4880-a561-69ea0016203d admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:31.664435 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110df4fd-deab-4880-a561-69ea0016203d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:31.667939 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110df4fd-deab-4880-a561-69ea0016203d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:31.744905 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 873/1747] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:26:31 2022] POST /identity/v3/users => generated 290 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:26:31.745694 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1283d6-968c-4a44-9836-2a09c02aeed7 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:31.758099 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1283d6-968c-4a44-9836-2a09c02aeed7 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:31.774128 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44907539-ee9d-447d-b088-e889fb38be34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:31.777491 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44907539-ee9d-447d-b088-e889fb38be34 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:31.777491 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44907539-ee9d-447d-b088-e889fb38be34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:31.777747 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44907539-ee9d-447d-b088-e889fb38be34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:31.780522 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44907539-ee9d-447d-b088-e889fb38be34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:31.785071 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44907539-ee9d-447d-b088-e889fb38be34 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:31.821320 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 875/1748] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 10:26:31.822742 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110df4fd-deab-4880-a561-69ea0016203d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:31.844183 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110df4fd-deab-4880-a561-69ea0016203d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:31.859289 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e90630-8ceb-4bbb-82e3-f6605b6cdeb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:31.860104 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e90630-8ceb-4bbb-82e3-f6605b6cdeb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:31.860394 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e90630-8ceb-4bbb-82e3-f6605b6cdeb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:31.860769 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e90630-8ceb-4bbb-82e3-f6605b6cdeb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:31.861556 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e90630-8ceb-4bbb-82e3-f6605b6cdeb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:31.863254 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e90630-8ceb-4bbb-82e3-f6605b6cdeb2 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:31.895508 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 874/1749] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:31.897868 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44907539-ee9d-447d-b088-e889fb38be34 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:31.913023 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44907539-ee9d-447d-b088-e889fb38be34 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:31.927231 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a558e32f-183b-44bc-a949-baff0aee45ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:31.928659 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558e32f-183b-44bc-a949-baff0aee45ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:31.929847 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558e32f-183b-44bc-a949-baff0aee45ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:31.929847 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558e32f-183b-44bc-a949-baff0aee45ab admin admin] PATH_INFO: `/v3/projects/5e4fe6f3cee8430da84f3bcbe6fdddb9/users/12f87b04ac254285aefa6a213153eee2/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:31.975659 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a558e32f-183b-44bc-a949-baff0aee45ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e4fe6f3cee8430da84f3bcbe6fdddb9, user_id=12f87b04ac254285aefa6a213153eee2, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:31.983625 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a558e32f-183b-44bc-a949-baff0aee45ab admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:32.019744 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 876/1750] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:26:31 2022] PUT /identity/v3/projects/5e4fe6f3cee8430da84f3bcbe6fdddb9/users/12f87b04ac254285aefa6a213153eee2/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:32.031116 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd3bd8-3cec-467b-8158-66efadafa235 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:32.031603 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd3bd8-3cec-467b-8158-66efadafa235 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:32.032068 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd3bd8-3cec-467b-8158-66efadafa235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:32.051459 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:32.109130 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-19cd3bd8-3cec-467b-8158-66efadafa235 None None] MFA Rules not processed for user `12f87b04ac254285aefa6a213153eee2`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:32.110354 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-19cd3bd8-3cec-467b-8158-66efadafa235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:32.294353 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 875/1751] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:32.300643 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a558e32f-183b-44bc-a949-baff0aee45ab admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:32.390799 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:32.582084 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a558e32f-183b-44bc-a949-baff0aee45ab admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:32.590568 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e926abf-f40d-4e14-9952-286c298eff4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:32.591569 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e926abf-f40d-4e14-9952-286c298eff4a admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:32.596253 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e926abf-f40d-4e14-9952-286c298eff4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:32.596712 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e926abf-f40d-4e14-9952-286c298eff4a admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:32.597721 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e926abf-f40d-4e14-9952-286c298eff4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:32.601486 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e926abf-f40d-4e14-9952-286c298eff4a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:32.674727 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 877/1752] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:26:32 2022] POST /identity/v3/projects => generated 375 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:26:32.675939 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cd3bd8-3cec-467b-8158-66efadafa235 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:32.700446 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cd3bd8-3cec-467b-8158-66efadafa235 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:32.863423 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c55de-a531-47e5-bf67-3b4348c64542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:32.864215 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c55de-a531-47e5-bf67-3b4348c64542 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:32.866025 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c55de-a531-47e5-bf67-3b4348c64542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:32.866405 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c55de-a531-47e5-bf67-3b4348c64542 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:32.869612 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983c55de-a531-47e5-bf67-3b4348c64542 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:32.872393 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983c55de-a531-47e5-bf67-3b4348c64542 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:32.905913 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c26dc-a44c-4368-94be-c00891f86f7f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:32.906444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c26dc-a44c-4368-94be-c00891f86f7f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:32.907017 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c26dc-a44c-4368-94be-c00891f86f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:32.968097 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 876/1753] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:26:32 2022] POST /identity/v3/users => generated 294 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:26:32.969418 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c55de-a531-47e5-bf67-3b4348c64542 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:32.982094 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c55de-a531-47e5-bf67-3b4348c64542 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:32.988840 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cf4f1b-b52b-4a8e-b2e4-857fb52a3799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:32.989640 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf4f1b-b52b-4a8e-b2e4-857fb52a3799 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:32.990019 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf4f1b-b52b-4a8e-b2e4-857fb52a3799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:32.990393 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf4f1b-b52b-4a8e-b2e4-857fb52a3799 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:32.991596 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cf4f1b-b52b-4a8e-b2e4-857fb52a3799 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:32.993718 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cf4f1b-b52b-4a8e-b2e4-857fb52a3799 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:33.006740 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-f10c26dc-a44c-4368-94be-c00891f86f7f None None] MFA Rules not processed for user `97f671de92d240ae85ca4893d1ff04e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:33.011086 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f10c26dc-a44c-4368-94be-c00891f86f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:33.011627 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 877/1754] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:33.012659 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cf4f1b-b52b-4a8e-b2e4-857fb52a3799 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:33.039037 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cf4f1b-b52b-4a8e-b2e4-857fb52a3799 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:33.045157 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d07547-7135-4efc-8875-2c354d9950fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:33.046007 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d07547-7135-4efc-8875-2c354d9950fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:33.046393 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d07547-7135-4efc-8875-2c354d9950fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:33.046807 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d07547-7135-4efc-8875-2c354d9950fb admin admin] PATH_INFO: `/v3/projects/c987d6c50c48457bb2e07546571159f9/users/9b471c86ff3b4c8ca8963b7316cbcc7c/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:33.116111 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d07547-7135-4efc-8875-2c354d9950fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c987d6c50c48457bb2e07546571159f9, user_id=9b471c86ff3b4c8ca8963b7316cbcc7c, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:33.122006 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d07547-7135-4efc-8875-2c354d9950fb admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:33.149444 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 878/1755] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:26:33 2022] PUT /identity/v3/projects/c987d6c50c48457bb2e07546571159f9/users/9b471c86ff3b4c8ca8963b7316cbcc7c/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:33.151645 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d07547-7135-4efc-8875-2c354d9950fb admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:33.162367 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-47d07547-7135-4efc-8875-2c354d9950fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:33.197226 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 878/1756] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:32 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:33.198261 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c26dc-a44c-4368-94be-c00891f86f7f None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:33.210063 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c26dc-a44c-4368-94be-c00891f86f7f None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:33.287094 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:33.326352 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b2682-af6b-4dfb-b794-cf3f29d84590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:33.327390 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b2682-af6b-4dfb-b794-cf3f29d84590 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:33.330952 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b2682-af6b-4dfb-b794-cf3f29d84590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:33.331695 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b2682-af6b-4dfb-b794-cf3f29d84590 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:33.332693 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906b2682-af6b-4dfb-b794-cf3f29d84590 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:33.334519 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d07547-7135-4efc-8875-2c354d9950fb admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:33.341288 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906b2682-af6b-4dfb-b794-cf3f29d84590 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:33.348416 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f48b4e-4eab-4911-90fb-17e9a8796652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:33.349100 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f48b4e-4eab-4911-90fb-17e9a8796652 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:33.349383 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f48b4e-4eab-4911-90fb-17e9a8796652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:33.349660 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f48b4e-4eab-4911-90fb-17e9a8796652 admin admin] PATH_INFO: `/v3/users/dc2c9f055929440d8e9645cdf9bebec8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:33.350853 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f48b4e-4eab-4911-90fb-17e9a8796652 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc2c9f055929440d8e9645cdf9bebec8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:33.352861 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f48b4e-4eab-4911-90fb-17e9a8796652 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:33.371243 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 879/1757] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:33.374021 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b2682-af6b-4dfb-b794-cf3f29d84590 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:33.379799 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-906b2682-af6b-4dfb-b794-cf3f29d84590 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:33.443563 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-39f48b4e-4eab-4911-90fb-17e9a8796652 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc2c9f055929440d8e9645cdf9bebec8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:33.444160 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-39f48b4e-4eab-4911-90fb-17e9a8796652 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc2c9f055929440d8e9645cdf9bebec8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:33.445319 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-39f48b4e-4eab-4911-90fb-17e9a8796652 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc2c9f055929440d8e9645cdf9bebec8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:33.464765 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-39f48b4e-4eab-4911-90fb-17e9a8796652 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc2c9f055929440d8e9645cdf9bebec8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:33.492378 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-39f48b4e-4eab-4911-90fb-17e9a8796652 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc2c9f055929440d8e9645cdf9bebec8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:33.503890 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:33.535963 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 879/1758] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:26:33 2022] DELETE /identity/v3/users/dc2c9f055929440d8e9645cdf9bebec8 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:33.537624 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f48b4e-4eab-4911-90fb-17e9a8796652 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:33.539455 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-39f48b4e-4eab-4911-90fb-17e9a8796652 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:33.584638 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b2682-af6b-4dfb-b794-cf3f29d84590 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:33.670919 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f48b4e-4eab-4911-90fb-17e9a8796652 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:33.679901 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d2bf38-04de-4409-9b8c-d1e5fa4e19a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:33.680980 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2bf38-04de-4409-9b8c-d1e5fa4e19a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:33.681360 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2bf38-04de-4409-9b8c-d1e5fa4e19a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:33.681731 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2bf38-04de-4409-9b8c-d1e5fa4e19a5 admin admin] PATH_INFO: `/v3/projects/c987d6c50c48457bb2e07546571159f9/users/9b471c86ff3b4c8ca8963b7316cbcc7c/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:33.691864 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d2bf38-04de-4409-9b8c-d1e5fa4e19a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c987d6c50c48457bb2e07546571159f9, user_id=9b471c86ff3b4c8ca8963b7316cbcc7c, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:33.698438 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d2bf38-04de-4409-9b8c-d1e5fa4e19a5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:33.724850 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 880/1759] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:26:33 2022] PUT /identity/v3/projects/c987d6c50c48457bb2e07546571159f9/users/9b471c86ff3b4c8ca8963b7316cbcc7c/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:33.737037 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70716ed0-0f40-47d9-b13c-4d8f91e278cb None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:33.737415 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70716ed0-0f40-47d9-b13c-4d8f91e278cb None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:33.738206 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70716ed0-0f40-47d9-b13c-4d8f91e278cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:33.763592 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd1a9e-bab1-4d76-8ae4-c9faa4f0cf1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:33.765314 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd1a9e-bab1-4d76-8ae4-c9faa4f0cf1b service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:33.767064 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd1a9e-bab1-4d76-8ae4-c9faa4f0cf1b service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:33.767557 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd1a9e-bab1-4d76-8ae4-c9faa4f0cf1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:33.774513 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-10dd1a9e-bab1-4d76-8ae4-c9faa4f0cf1b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:33.806507 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-70716ed0-0f40-47d9-b13c-4d8f91e278cb None None] MFA Rules not processed for user `9b471c86ff3b4c8ca8963b7316cbcc7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:33.810124 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-70716ed0-0f40-47d9-b13c-4d8f91e278cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:33.905165 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:33.907558 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dd1a9e-bab1-4d76-8ae4-c9faa4f0cf1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:33.909207 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dd1a9e-bab1-4d76-8ae4-c9faa4f0cf1b service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:33.929130 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 880/1760] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:26:33 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:26:33.960681 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 881/1761] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:33 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:33.973302 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd1a9e-bab1-4d76-8ae4-c9faa4f0cf1b service nova] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:33.975295 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-10dd1a9e-bab1-4d76-8ae4-c9faa4f0cf1b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:34.107678 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd1a9e-bab1-4d76-8ae4-c9faa4f0cf1b service nova] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:34.171213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70716ed0-0f40-47d9-b13c-4d8f91e278cb None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:34.172698 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-70716ed0-0f40-47d9-b13c-4d8f91e278cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:34.205831 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:34.264774 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35406402-a9a8-48c2-b522-b067dc897e29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:34.265412 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35406402-a9a8-48c2-b522-b067dc897e29 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:34.265694 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35406402-a9a8-48c2-b522-b067dc897e29 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:34.265962 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35406402-a9a8-48c2-b522-b067dc897e29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:34.290138 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35406402-a9a8-48c2-b522-b067dc897e29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:34.291746 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35406402-a9a8-48c2-b522-b067dc897e29 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:34.308770 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 881/1762] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:26:33 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:26:34.352226 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70716ed0-0f40-47d9-b13c-4d8f91e278cb None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:34.360512 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35406402-a9a8-48c2-b522-b067dc897e29 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:34.362625 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-35406402-a9a8-48c2-b522-b067dc897e29 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:34.365077 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e411a-4622-4cd4-9403-d7d40646a830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:34.365754 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e411a-4622-4cd4-9403-d7d40646a830 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:34.366043 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e411a-4622-4cd4-9403-d7d40646a830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:34.366323 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e411a-4622-4cd4-9403-d7d40646a830 admin admin] PATH_INFO: `/v3/projects/c8cb7b6044e0420283e0cb1abbbb1ed2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:34.367633 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4e411a-4622-4cd4-9403-d7d40646a830 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8cb7b6044e0420283e0cb1abbbb1ed2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:34.370472 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4e411a-4622-4cd4-9403-d7d40646a830 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:34.382784 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35406402-a9a8-48c2-b522-b067dc897e29 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:34.390779 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93965fce-1d13-4264-a2cc-7b9358e19d98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:34.391459 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93965fce-1d13-4264-a2cc-7b9358e19d98 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:34.391901 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93965fce-1d13-4264-a2cc-7b9358e19d98 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:34.392218 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93965fce-1d13-4264-a2cc-7b9358e19d98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:34.396159 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-93965fce-1d13-4264-a2cc-7b9358e19d98 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:34.488180 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-aa4e411a-4622-4cd4-9403-d7d40646a830 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8cb7b6044e0420283e0cb1abbbb1ed2', 'request_id': 'req-aa4e411a-4622-4cd4-9403-d7d40646a830'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:34.502067 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-aa4e411a-4622-4cd4-9403-d7d40646a830 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8cb7b6044e0420283e0cb1abbbb1ed2', 'request_id': 'req-aa4e411a-4622-4cd4-9403-d7d40646a830'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:34.511960 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-aa4e411a-4622-4cd4-9403-d7d40646a830 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8cb7b6044e0420283e0cb1abbbb1ed2', 'request_id': 'req-aa4e411a-4622-4cd4-9403-d7d40646a830'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:34.521740 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93965fce-1d13-4264-a2cc-7b9358e19d98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:34.524145 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93965fce-1d13-4264-a2cc-7b9358e19d98 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:34.527969 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-aa4e411a-4622-4cd4-9403-d7d40646a830 admin admin] The token cache is being invalidate because project c8cb7b6044e0420283e0cb1abbbb1ed2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:26:34.528731 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-aa4e411a-4622-4cd4-9403-d7d40646a830 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:34.529586 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-aa4e411a-4622-4cd4-9403-d7d40646a830 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:34.531839 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 882/1763] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:26:34 2022] DELETE /identity/v3/projects/c8cb7b6044e0420283e0cb1abbbb1ed2 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:26:34.533546 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbf40f-e5a2-49e3-8563-b7147681f902 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:34.534009 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbf40f-e5a2-49e3-8563-b7147681f902 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:34.534494 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbf40f-e5a2-49e3-8563-b7147681f902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:34.558476 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-3dfbf40f-e5a2-49e3-8563-b7147681f902 None None] MFA Rules not processed for user `7e7eb355fac04252a4be1f8e60037924`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:34.560297 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3dfbf40f-e5a2-49e3-8563-b7147681f902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:34.563275 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 882/1764] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:26:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 10:26:34.566536 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93965fce-1d13-4264-a2cc-7b9358e19d98 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:34.600400 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93965fce-1d13-4264-a2cc-7b9358e19d98 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:34.676924 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 883/1765] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:34.779018 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb40b1-7ca5-40b3-82be-c39926235105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:34.779697 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb40b1-7ca5-40b3-82be-c39926235105 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:34.782439 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb40b1-7ca5-40b3-82be-c39926235105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:34.783394 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb40b1-7ca5-40b3-82be-c39926235105 admin admin] PATH_INFO: `/v3/users/24386bd3e3c9484f98a57ad1546a73ce` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:34.787633 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefb40b1-7ca5-40b3-82be-c39926235105 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24386bd3e3c9484f98a57ad1546a73ce)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:34.789621 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefb40b1-7ca5-40b3-82be-c39926235105 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:34.865497 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-eefb40b1-7ca5-40b3-82be-c39926235105 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24386bd3e3c9484f98a57ad1546a73ce'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:34.879207 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-eefb40b1-7ca5-40b3-82be-c39926235105 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24386bd3e3c9484f98a57ad1546a73ce'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:34.880807 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-eefb40b1-7ca5-40b3-82be-c39926235105 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24386bd3e3c9484f98a57ad1546a73ce'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:34.907709 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-eefb40b1-7ca5-40b3-82be-c39926235105 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24386bd3e3c9484f98a57ad1546a73ce'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:34.909790 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-eefb40b1-7ca5-40b3-82be-c39926235105 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24386bd3e3c9484f98a57ad1546a73ce'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:34.920005 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:34.958672 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 883/1766] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:26:34 2022] DELETE /identity/v3/users/24386bd3e3c9484f98a57ad1546a73ce => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:35.626120 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfbf40f-e5a2-49e3-8563-b7147681f902 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:35.628271 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3dfbf40f-e5a2-49e3-8563-b7147681f902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:35.670231 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212abc82-d617-4269-a61d-7229694bbe04 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:35.670417 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212abc82-d617-4269-a61d-7229694bbe04 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:35.670699 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212abc82-d617-4269-a61d-7229694bbe04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:35.679268 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfbf40f-e5a2-49e3-8563-b7147681f902 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:35.711650 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-212abc82-d617-4269-a61d-7229694bbe04 None None] MFA Rules not processed for user `a7deb94069a449269cb03406232174d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:35.714579 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-212abc82-d617-4269-a61d-7229694bbe04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:35.732525 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:35.811240 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411bc980-45eb-47db-82b8-13f29ba482cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:35.812649 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411bc980-45eb-47db-82b8-13f29ba482cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:35.813128 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411bc980-45eb-47db-82b8-13f29ba482cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:35.813845 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411bc980-45eb-47db-82b8-13f29ba482cf admin admin] PATH_INFO: `/v3/projects/c0ed0d86ce8347ee9ee421ce0730af5c` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:35.815997 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411bc980-45eb-47db-82b8-13f29ba482cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0ed0d86ce8347ee9ee421ce0730af5c)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:35.818825 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411bc980-45eb-47db-82b8-13f29ba482cf admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:35.937288 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 884/1767] 10.222.0.36 () {58 vars in 1015 bytes} [Fri Aug 5 10:26:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:26:35.961898 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-411bc980-45eb-47db-82b8-13f29ba482cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0ed0d86ce8347ee9ee421ce0730af5c', 'request_id': 'req-411bc980-45eb-47db-82b8-13f29ba482cf'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:35.973137 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-411bc980-45eb-47db-82b8-13f29ba482cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0ed0d86ce8347ee9ee421ce0730af5c', 'request_id': 'req-411bc980-45eb-47db-82b8-13f29ba482cf'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:35.992373 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-411bc980-45eb-47db-82b8-13f29ba482cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0ed0d86ce8347ee9ee421ce0730af5c', 'request_id': 'req-411bc980-45eb-47db-82b8-13f29ba482cf'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:36.032768 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-411bc980-45eb-47db-82b8-13f29ba482cf admin admin] The token cache is being invalidate because project c0ed0d86ce8347ee9ee421ce0730af5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:26:36.033052 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-411bc980-45eb-47db-82b8-13f29ba482cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:36.034001 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-411bc980-45eb-47db-82b8-13f29ba482cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:36.039711 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 884/1768] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:26:35 2022] DELETE /identity/v3/projects/c0ed0d86ce8347ee9ee421ce0730af5c => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:36.044664 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a47b94-6978-4bdf-91ac-f7ac4f5435be None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:36.045151 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a47b94-6978-4bdf-91ac-f7ac4f5435be None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:36.045562 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a47b94-6978-4bdf-91ac-f7ac4f5435be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:36.116557 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-19a47b94-6978-4bdf-91ac-f7ac4f5435be None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:36.124831 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-19a47b94-6978-4bdf-91ac-f7ac4f5435be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:36.325677 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 885/1769] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:36.334872 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba6ffec-face-4abf-9c86-19e1e4462013 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:36.335191 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba6ffec-face-4abf-9c86-19e1e4462013 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:36.335468 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba6ffec-face-4abf-9c86-19e1e4462013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:36.373679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9ba6ffec-face-4abf-9c86-19e1e4462013 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:36.378508 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9ba6ffec-face-4abf-9c86-19e1e4462013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:36.395266 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a47b94-6978-4bdf-91ac-f7ac4f5435be None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:36.396651 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-19a47b94-6978-4bdf-91ac-f7ac4f5435be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:36.413147 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 885/1770] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:36.414209 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba6ffec-face-4abf-9c86-19e1e4462013 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:36.444870 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba6ffec-face-4abf-9c86-19e1e4462013 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:36.459206 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167afb66-0eb0-4d54-bcde-1be149a5cf85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:36.459854 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167afb66-0eb0-4d54-bcde-1be149a5cf85 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:36.460208 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167afb66-0eb0-4d54-bcde-1be149a5cf85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:36.461569 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167afb66-0eb0-4d54-bcde-1be149a5cf85 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:36.462362 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167afb66-0eb0-4d54-bcde-1be149a5cf85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:26:36.464020 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167afb66-0eb0-4d54-bcde-1be149a5cf85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:36.465945 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167afb66-0eb0-4d54-bcde-1be149a5cf85 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:36.528715 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 886/1771] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:26:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:26:36.551660 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167afb66-0eb0-4d54-bcde-1be149a5cf85 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:36.584362 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167afb66-0eb0-4d54-bcde-1be149a5cf85 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:36.606616 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a47b94-6978-4bdf-91ac-f7ac4f5435be None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:36.617321 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed788330-ae01-4f62-8d64-75ce95c0150e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:36.617987 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed788330-ae01-4f62-8d64-75ce95c0150e admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:36.620767 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed788330-ae01-4f62-8d64-75ce95c0150e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:36.621042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed788330-ae01-4f62-8d64-75ce95c0150e admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:36.625137 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e7526f-92d4-4840-872a-770004e7ed0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:36.625367 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e7526f-92d4-4840-872a-770004e7ed0c service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:36.625652 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e7526f-92d4-4840-872a-770004e7ed0c service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:36.625925 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e7526f-92d4-4840-872a-770004e7ed0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:36.626837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed788330-ae01-4f62-8d64-75ce95c0150e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:36.635417 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed788330-ae01-4f62-8d64-75ce95c0150e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:36.641139 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b0e7526f-92d4-4840-872a-770004e7ed0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:36.689501 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e7526f-92d4-4840-872a-770004e7ed0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:36.692581 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e7526f-92d4-4840-872a-770004e7ed0c service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:36.731781 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 886/1772] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:26:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 10:26:36.755480 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 887/1773] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:26:36 2022] POST /identity/v3/projects => generated 399 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:26:36.756504 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e7526f-92d4-4840-872a-770004e7ed0c service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:36.774616 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e7526f-92d4-4840-872a-770004e7ed0c service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:36.877243 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:36.908383 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f24ee6-1637-4d78-919d-cc34f19bb9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:36.908383 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f24ee6-1637-4d78-919d-cc34f19bb9c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:36.908765 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f24ee6-1637-4d78-919d-cc34f19bb9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:36.909165 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f24ee6-1637-4d78-919d-cc34f19bb9c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:36.909951 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f24ee6-1637-4d78-919d-cc34f19bb9c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:36.913028 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f24ee6-1637-4d78-919d-cc34f19bb9c7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:36.961703 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 887/1774] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:26:36 2022] POST /identity/v3/users => generated 300 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:26:36.965001 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed788330-ae01-4f62-8d64-75ce95c0150e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:36.989667 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed788330-ae01-4f62-8d64-75ce95c0150e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:37.000065 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc5a73a-b9b9-4b46-b913-d053897c829a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:37.001860 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc5a73a-b9b9-4b46-b913-d053897c829a admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:37.001860 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc5a73a-b9b9-4b46-b913-d053897c829a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:37.001860 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc5a73a-b9b9-4b46-b913-d053897c829a admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:37.002769 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc5a73a-b9b9-4b46-b913-d053897c829a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:37.015274 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc5a73a-b9b9-4b46-b913-d053897c829a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:37.049961 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 888/1775] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:36 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 10:26:37.054869 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f24ee6-1637-4d78-919d-cc34f19bb9c7 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:37.077358 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f24ee6-1637-4d78-919d-cc34f19bb9c7 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:37.099738 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab91e9b-0132-4b88-9e7a-df8968ef8ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:37.103842 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab91e9b-0132-4b88-9e7a-df8968ef8ae0 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:37.106739 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab91e9b-0132-4b88-9e7a-df8968ef8ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:37.108476 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab91e9b-0132-4b88-9e7a-df8968ef8ae0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:37.111819 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab91e9b-0132-4b88-9e7a-df8968ef8ae0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:37.119626 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab91e9b-0132-4b88-9e7a-df8968ef8ae0 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:37.175093 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc5a73a-b9b9-4b46-b913-d053897c829a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:37.189077 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 888/1776] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:37 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:37.205173 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc5a73a-b9b9-4b46-b913-d053897c829a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:37.224812 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c1b2b9-d268-4f82-b89d-c72ba3d3743e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:37.226426 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c1b2b9-d268-4f82-b89d-c72ba3d3743e admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:37.226787 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c1b2b9-d268-4f82-b89d-c72ba3d3743e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:37.227004 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c1b2b9-d268-4f82-b89d-c72ba3d3743e admin admin] PATH_INFO: `/v3/projects/ecf2abd1c72c4e59a4567919abb3181d/users/a1356b70ac6d48e297fa8e5340b80274/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:37.282410 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c1b2b9-d268-4f82-b89d-c72ba3d3743e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecf2abd1c72c4e59a4567919abb3181d, user_id=a1356b70ac6d48e297fa8e5340b80274, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:37.308378 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c1b2b9-d268-4f82-b89d-c72ba3d3743e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:37.379951 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4010db-68e8-45a5-89c9-68a3730e0400 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:37.380166 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 889/1777] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:26:37 2022] PUT /identity/v3/projects/ecf2abd1c72c4e59a4567919abb3181d/users/a1356b70ac6d48e297fa8e5340b80274/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:37.380780 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4010db-68e8-45a5-89c9-68a3730e0400 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:37.381171 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4010db-68e8-45a5-89c9-68a3730e0400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:37.494808 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-8e4010db-68e8-45a5-89c9-68a3730e0400 None None] MFA Rules not processed for user `a1356b70ac6d48e297fa8e5340b80274`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:37.501868 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8e4010db-68e8-45a5-89c9-68a3730e0400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:37.698576 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 889/1778] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:37 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:37.703087 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c1b2b9-d268-4f82-b89d-c72ba3d3743e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:37.816212 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:37.876561 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c1b2b9-d268-4f82-b89d-c72ba3d3743e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:37.889194 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31953f8f-5ef2-4f28-84c8-dba4fbc1c452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:37.889952 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31953f8f-5ef2-4f28-84c8-dba4fbc1c452 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:37.890303 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31953f8f-5ef2-4f28-84c8-dba4fbc1c452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:37.890506 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31953f8f-5ef2-4f28-84c8-dba4fbc1c452 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:37.892657 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31953f8f-5ef2-4f28-84c8-dba4fbc1c452 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:37.897688 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31953f8f-5ef2-4f28-84c8-dba4fbc1c452 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:37.956492 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 890/1779] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:26:37 2022] POST /identity/v3/projects => generated 399 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:26:37.960896 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4010db-68e8-45a5-89c9-68a3730e0400 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:37.976917 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4010db-68e8-45a5-89c9-68a3730e0400 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:38.212591 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e9ae29-c574-41cd-9066-7ec8a9ecce42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:38.213529 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9ae29-c574-41cd-9066-7ec8a9ecce42 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:38.213964 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9ae29-c574-41cd-9066-7ec8a9ecce42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:38.214368 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9ae29-c574-41cd-9066-7ec8a9ecce42 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:38.218175 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e9ae29-c574-41cd-9066-7ec8a9ecce42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:38.225288 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e9ae29-c574-41cd-9066-7ec8a9ecce42 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:38.304320 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 890/1780] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:26:37 2022] POST /identity/v3/users => generated 306 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:26:38.308269 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31953f8f-5ef2-4f28-84c8-dba4fbc1c452 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:38.329301 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31953f8f-5ef2-4f28-84c8-dba4fbc1c452 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:38.341542 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506453c2-b228-4b35-b565-f4864aaa2762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:38.342212 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506453c2-b228-4b35-b565-f4864aaa2762 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:38.342487 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506453c2-b228-4b35-b565-f4864aaa2762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:38.342765 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506453c2-b228-4b35-b565-f4864aaa2762 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:38.344078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506453c2-b228-4b35-b565-f4864aaa2762 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:38.345351 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506453c2-b228-4b35-b565-f4864aaa2762 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:38.370500 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 891/1781] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:38 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:38.374708 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e9ae29-c574-41cd-9066-7ec8a9ecce42 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:38.393597 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e9ae29-c574-41cd-9066-7ec8a9ecce42 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:38.412288 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9962eb1-3643-4db8-aa9c-7d71cc48beb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:38.423545 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9962eb1-3643-4db8-aa9c-7d71cc48beb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:38.423879 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9962eb1-3643-4db8-aa9c-7d71cc48beb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:38.424311 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9962eb1-3643-4db8-aa9c-7d71cc48beb5 admin admin] PATH_INFO: `/v3/projects/0a921238708249f9865c942fdd1182d4/users/93b9e102c7964954a2cacaac1ea7440a/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:38.477335 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9962eb1-3643-4db8-aa9c-7d71cc48beb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a921238708249f9865c942fdd1182d4, user_id=93b9e102c7964954a2cacaac1ea7440a, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:38.481804 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9962eb1-3643-4db8-aa9c-7d71cc48beb5 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:38.543369 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 891/1782] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:26:38 2022] PUT /identity/v3/projects/0a921238708249f9865c942fdd1182d4/users/93b9e102c7964954a2cacaac1ea7440a/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:38.544269 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506453c2-b228-4b35-b565-f4864aaa2762 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:38.569360 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506453c2-b228-4b35-b565-f4864aaa2762 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:38.704579 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d24cfde-6ced-4767-8e43-f22b7590c846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:38.706193 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d24cfde-6ced-4767-8e43-f22b7590c846 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:38.706193 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d24cfde-6ced-4767-8e43-f22b7590c846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:38.706193 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d24cfde-6ced-4767-8e43-f22b7590c846 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:38.706713 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d24cfde-6ced-4767-8e43-f22b7590c846 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:38.708408 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d24cfde-6ced-4767-8e43-f22b7590c846 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:38.758298 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 892/1783] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:38 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:38.761510 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9962eb1-3643-4db8-aa9c-7d71cc48beb5 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:38.777342 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9962eb1-3643-4db8-aa9c-7d71cc48beb5 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:38.789677 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4be4136-5414-457d-92e1-9d479484ff04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:38.791170 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be4136-5414-457d-92e1-9d479484ff04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:38.792126 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be4136-5414-457d-92e1-9d479484ff04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:38.795906 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be4136-5414-457d-92e1-9d479484ff04 admin admin] PATH_INFO: `/v3/projects/0a921238708249f9865c942fdd1182d4/users/93b9e102c7964954a2cacaac1ea7440a/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:38.800215 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4be4136-5414-457d-92e1-9d479484ff04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a921238708249f9865c942fdd1182d4, user_id=93b9e102c7964954a2cacaac1ea7440a, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:38.810705 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4be4136-5414-457d-92e1-9d479484ff04 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:38.838851 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 892/1784] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:26:38 2022] PUT /identity/v3/projects/0a921238708249f9865c942fdd1182d4/users/93b9e102c7964954a2cacaac1ea7440a/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:38.857921 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdea501-afe4-4748-b968-9ae00a4184c7 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:38.858203 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdea501-afe4-4748-b968-9ae00a4184c7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:38.858694 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdea501-afe4-4748-b968-9ae00a4184c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:38.932970 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-dfdea501-afe4-4748-b968-9ae00a4184c7 None None] MFA Rules not processed for user `93b9e102c7964954a2cacaac1ea7440a`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:38.934584 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-dfdea501-afe4-4748-b968-9ae00a4184c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:39.197803 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 893/1785] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:38 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:39.209926 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4be4136-5414-457d-92e1-9d479484ff04 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:39.212464 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e4be4136-5414-457d-92e1-9d479484ff04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:39.232612 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:39.546582 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4be4136-5414-457d-92e1-9d479484ff04 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:39.563488 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bb3a8a-8e43-48e2-addf-370cf8e08f7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:39.564106 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb3a8a-8e43-48e2-addf-370cf8e08f7b service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:39.564386 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb3a8a-8e43-48e2-addf-370cf8e08f7b service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:39.564663 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb3a8a-8e43-48e2-addf-370cf8e08f7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:39.617463 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bb3a8a-8e43-48e2-addf-370cf8e08f7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:39.620486 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bb3a8a-8e43-48e2-addf-370cf8e08f7b service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:39.646601 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 893/1786] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:26:39 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:26:39.831149 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdea501-afe4-4748-b968-9ae00a4184c7 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:39.856971 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdea501-afe4-4748-b968-9ae00a4184c7 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:39.892363 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a5f6c0-9e1e-49e0-9aa8-8e7009ceeaeb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:39.893598 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5f6c0-9e1e-49e0-9aa8-8e7009ceeaeb service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:39.894126 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5f6c0-9e1e-49e0-9aa8-8e7009ceeaeb service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:39.894583 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5f6c0-9e1e-49e0-9aa8-8e7009ceeaeb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:40.009247 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:40.085771 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a5f6c0-9e1e-49e0-9aa8-8e7009ceeaeb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:40.088900 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a5f6c0-9e1e-49e0-9aa8-8e7009ceeaeb service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:40.139115 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 894/1787] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:26:39 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:26:40.142109 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bb3a8a-8e43-48e2-addf-370cf8e08f7b service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:40.143888 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c3bb3a8a-8e43-48e2-addf-370cf8e08f7b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:40.353522 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bb3a8a-8e43-48e2-addf-370cf8e08f7b service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:40.366646 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba2a0d-324b-4463-bf5b-9b15f65484b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:40.370957 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba2a0d-324b-4463-bf5b-9b15f65484b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:40.371518 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba2a0d-324b-4463-bf5b-9b15f65484b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:40.371995 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba2a0d-324b-4463-bf5b-9b15f65484b0 admin admin] PATH_INFO: `/v3/users/8b221fdbe86f45b7ae929c1f9b4af448` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:40.373980 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ba2a0d-324b-4463-bf5b-9b15f65484b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b221fdbe86f45b7ae929c1f9b4af448)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:40.376282 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ba2a0d-324b-4463-bf5b-9b15f65484b0 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:40.573935 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a7ba2a0d-324b-4463-bf5b-9b15f65484b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b221fdbe86f45b7ae929c1f9b4af448'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:40.604552 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a7ba2a0d-324b-4463-bf5b-9b15f65484b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b221fdbe86f45b7ae929c1f9b4af448'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:40.605227 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a7ba2a0d-324b-4463-bf5b-9b15f65484b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b221fdbe86f45b7ae929c1f9b4af448'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:40.671726 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a7ba2a0d-324b-4463-bf5b-9b15f65484b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b221fdbe86f45b7ae929c1f9b4af448'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:40.673026 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a7ba2a0d-324b-4463-bf5b-9b15f65484b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b221fdbe86f45b7ae929c1f9b4af448'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:40.715516 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 894/1788] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:26:40 2022] DELETE /identity/v3/users/8b221fdbe86f45b7ae929c1f9b4af448 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:40.722744 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34e3a5-2d58-4151-ad19-c1ce7a677d1c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:40.723877 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34e3a5-2d58-4151-ad19-c1ce7a677d1c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:40.724416 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34e3a5-2d58-4151-ad19-c1ce7a677d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:40.764999 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1e34e3a5-2d58-4151-ad19-c1ce7a677d1c None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:40.769421 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1e34e3a5-2d58-4151-ad19-c1ce7a677d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:40.853813 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 895/1789] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:41.184103 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba2a0d-324b-4463-bf5b-9b15f65484b0 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:41.185623 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a7ba2a0d-324b-4463-bf5b-9b15f65484b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:41.267278 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:41.386932 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba2a0d-324b-4463-bf5b-9b15f65484b0 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:41.432445 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb65e442-37f9-4f4a-a146-26e10e7b6cf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:41.438145 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65e442-37f9-4f4a-a146-26e10e7b6cf0 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:41.440906 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65e442-37f9-4f4a-a146-26e10e7b6cf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:41.442632 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65e442-37f9-4f4a-a146-26e10e7b6cf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:41.477706 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb65e442-37f9-4f4a-a146-26e10e7b6cf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:41.485599 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb65e442-37f9-4f4a-a146-26e10e7b6cf0 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:41.523554 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 895/1790] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:26:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:26:42.238248 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290d4dd-60ac-4b52-8263-7da9e73d5b39 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:42.239497 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290d4dd-60ac-4b52-8263-7da9e73d5b39 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:42.239994 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290d4dd-60ac-4b52-8263-7da9e73d5b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:42.298181 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6290d4dd-60ac-4b52-8263-7da9e73d5b39 None None] MFA Rules not processed for user `97f671de92d240ae85ca4893d1ff04e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:42.301651 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6290d4dd-60ac-4b52-8263-7da9e73d5b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:42.378353 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23229ae-acb6-4fbf-8946-a87f24f3e16f None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:42.378630 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23229ae-acb6-4fbf-8946-a87f24f3e16f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:42.379126 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23229ae-acb6-4fbf-8946-a87f24f3e16f None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:42.381345 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 896/1791] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 5 10:26:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:26:42.464174 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 896/1792] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:42 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:42.561356 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e790709-ce29-42b5-8aee-3e262746ff8d None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:42.562715 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e790709-ce29-42b5-8aee-3e262746ff8d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:42.563756 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e790709-ce29-42b5-8aee-3e262746ff8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:42.645970 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-5e790709-ce29-42b5-8aee-3e262746ff8d None None] MFA Rules not processed for user `b938f1f41db94b0eafe95bc8eeaae175`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:42.647234 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5e790709-ce29-42b5-8aee-3e262746ff8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:42.780699 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 897/1793] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 5 10:26:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:26:44.382747 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f155f-9210-45a0-94e6-bbda3af0b408 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:44.383304 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f155f-9210-45a0-94e6-bbda3af0b408 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:44.383304 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f155f-9210-45a0-94e6-bbda3af0b408 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:44.387584 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 897/1794] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 5 10:26:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:26:44.423114 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723f155f-9210-45a0-94e6-bbda3af0b408 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:44.424667 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc18a462-9015-4584-bd51-23947321ca0d None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:44.427146 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc18a462-9015-4584-bd51-23947321ca0d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:44.428311 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-723f155f-9210-45a0-94e6-bbda3af0b408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:44.429087 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc18a462-9015-4584-bd51-23947321ca0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:44.490870 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723f155f-9210-45a0-94e6-bbda3af0b408 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:44.497523 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-dc18a462-9015-4584-bd51-23947321ca0d None None] MFA Rules not processed for user `b938f1f41db94b0eafe95bc8eeaae175`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:44.507114 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a4a53a-603a-4dad-ae5c-a3f997cd9361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:44.507427 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-dc18a462-9015-4584-bd51-23947321ca0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:44.509217 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a4a53a-603a-4dad-ae5c-a3f997cd9361 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:44.509458 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a4a53a-603a-4dad-ae5c-a3f997cd9361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:44.510024 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a4a53a-603a-4dad-ae5c-a3f997cd9361 admin admin] PATH_INFO: `/v3/projects/ca2bf86444e54a5abf161b8161a015a4` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:44.513258 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a4a53a-603a-4dad-ae5c-a3f997cd9361 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca2bf86444e54a5abf161b8161a015a4)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:44.516227 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a4a53a-603a-4dad-ae5c-a3f997cd9361 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:44.532401 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 898/1795] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 5 10:26:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:44.533482 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc18a462-9015-4584-bd51-23947321ca0d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:44.535205 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-dc18a462-9015-4584-bd51-23947321ca0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:44.587508 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc18a462-9015-4584-bd51-23947321ca0d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:44.601376 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede76a0-5e20-4b87-addc-71ebc7914f6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:44.602400 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede76a0-5e20-4b87-addc-71ebc7914f6b service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:44.604917 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede76a0-5e20-4b87-addc-71ebc7914f6b service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:44.605459 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede76a0-5e20-4b87-addc-71ebc7914f6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:44.638805 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ede76a0-5e20-4b87-addc-71ebc7914f6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:44.642498 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ede76a0-5e20-4b87-addc-71ebc7914f6b service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:44.663199 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-83a4a53a-603a-4dad-ae5c-a3f997cd9361 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca2bf86444e54a5abf161b8161a015a4', 'request_id': 'req-83a4a53a-603a-4dad-ae5c-a3f997cd9361'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:44.682544 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 899/1796] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:26:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:26:44.696399 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-83a4a53a-603a-4dad-ae5c-a3f997cd9361 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca2bf86444e54a5abf161b8161a015a4', 'request_id': 'req-83a4a53a-603a-4dad-ae5c-a3f997cd9361'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:44.722518 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-83a4a53a-603a-4dad-ae5c-a3f997cd9361 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca2bf86444e54a5abf161b8161a015a4', 'request_id': 'req-83a4a53a-603a-4dad-ae5c-a3f997cd9361'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:44.730512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:44.827105 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-83a4a53a-603a-4dad-ae5c-a3f997cd9361 admin admin] The token cache is being invalidate because project ca2bf86444e54a5abf161b8161a015a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:26:44.827644 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-83a4a53a-603a-4dad-ae5c-a3f997cd9361 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:44.835633 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-83a4a53a-603a-4dad-ae5c-a3f997cd9361 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:44.845854 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 898/1797] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:26:44 2022] DELETE /identity/v3/projects/ca2bf86444e54a5abf161b8161a015a4 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:44.849733 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede76a0-5e20-4b87-addc-71ebc7914f6b service nova] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:44.875902 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede76a0-5e20-4b87-addc-71ebc7914f6b service nova] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:45.096681 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85809466-88b1-4f1c-9a64-f399d7db4405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:45.097383 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85809466-88b1-4f1c-9a64-f399d7db4405 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:45.097814 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85809466-88b1-4f1c-9a64-f399d7db4405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:45.098107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85809466-88b1-4f1c-9a64-f399d7db4405 admin admin] PATH_INFO: `/v3/users/12f87b04ac254285aefa6a213153eee2` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:45.102159 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85809466-88b1-4f1c-9a64-f399d7db4405 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12f87b04ac254285aefa6a213153eee2)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:45.104185 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85809466-88b1-4f1c-9a64-f399d7db4405 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:45.230440 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:45.246385 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-85809466-88b1-4f1c-9a64-f399d7db4405 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12f87b04ac254285aefa6a213153eee2'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:45.267780 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-85809466-88b1-4f1c-9a64-f399d7db4405 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12f87b04ac254285aefa6a213153eee2'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:45.269331 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-85809466-88b1-4f1c-9a64-f399d7db4405 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12f87b04ac254285aefa6a213153eee2'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:45.302053 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-85809466-88b1-4f1c-9a64-f399d7db4405 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12f87b04ac254285aefa6a213153eee2'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:45.305485 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-85809466-88b1-4f1c-9a64-f399d7db4405 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12f87b04ac254285aefa6a213153eee2'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:45.354590 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 900/1798] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:26:44 2022] DELETE /identity/v3/users/12f87b04ac254285aefa6a213153eee2 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:46.222255 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a4a53a-603a-4dad-ae5c-a3f997cd9361 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:46.225322 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-83a4a53a-603a-4dad-ae5c-a3f997cd9361 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:46.248686 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a4a53a-603a-4dad-ae5c-a3f997cd9361 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:46.454471 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a872b7d7-4d0c-49d8-a4a9-6ad7936c0e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:46.457810 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872b7d7-4d0c-49d8-a4a9-6ad7936c0e9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:46.459939 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872b7d7-4d0c-49d8-a4a9-6ad7936c0e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:46.460178 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872b7d7-4d0c-49d8-a4a9-6ad7936c0e9d admin admin] PATH_INFO: `/v3/users/a1356b70ac6d48e297fa8e5340b80274` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:46.464104 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a872b7d7-4d0c-49d8-a4a9-6ad7936c0e9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1356b70ac6d48e297fa8e5340b80274)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:46.471513 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a872b7d7-4d0c-49d8-a4a9-6ad7936c0e9d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:46.629552 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a872b7d7-4d0c-49d8-a4a9-6ad7936c0e9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1356b70ac6d48e297fa8e5340b80274'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:46.630725 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a872b7d7-4d0c-49d8-a4a9-6ad7936c0e9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1356b70ac6d48e297fa8e5340b80274'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:46.633860 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a872b7d7-4d0c-49d8-a4a9-6ad7936c0e9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1356b70ac6d48e297fa8e5340b80274'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:46.681789 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a872b7d7-4d0c-49d8-a4a9-6ad7936c0e9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1356b70ac6d48e297fa8e5340b80274'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:46.729167 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a872b7d7-4d0c-49d8-a4a9-6ad7936c0e9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1356b70ac6d48e297fa8e5340b80274'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:46.819653 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 899/1799] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:26:46 2022] DELETE /identity/v3/users/a1356b70ac6d48e297fa8e5340b80274 => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:46.821698 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffdaf84-9f5c-4176-881c-334e7acca9e3 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:46.822506 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffdaf84-9f5c-4176-881c-334e7acca9e3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:46.823232 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffdaf84-9f5c-4176-881c-334e7acca9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:46.872573 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-6ffdaf84-9f5c-4176-881c-334e7acca9e3 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:46.879681 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6ffdaf84-9f5c-4176-881c-334e7acca9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:47.016612 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 901/1800] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:47.036805 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a872b7d7-4d0c-49d8-a4a9-6ad7936c0e9d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:47.038268 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a872b7d7-4d0c-49d8-a4a9-6ad7936c0e9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:47.094036 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffdaf84-9f5c-4176-881c-334e7acca9e3 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:47.097702 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6ffdaf84-9f5c-4176-881c-334e7acca9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:47.168581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a872b7d7-4d0c-49d8-a4a9-6ad7936c0e9d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:47.172314 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d5be27-7bd3-403a-b859-edaa6a0ee827 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:47.173004 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d5be27-7bd3-403a-b859-edaa6a0ee827 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:47.173289 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d5be27-7bd3-403a-b859-edaa6a0ee827 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:47.173555 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d5be27-7bd3-403a-b859-edaa6a0ee827 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:47.195543 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d5be27-7bd3-403a-b859-edaa6a0ee827 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:47.197273 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d5be27-7bd3-403a-b859-edaa6a0ee827 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:47.222659 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 900/1801] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:26:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:26:47.238613 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffdaf84-9f5c-4176-881c-334e7acca9e3 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:47.245112 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4901e0a-2f07-4b44-81c9-8d029381163d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:47.245785 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4901e0a-2f07-4b44-81c9-8d029381163d service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:47.246022 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4901e0a-2f07-4b44-81c9-8d029381163d service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:47.246294 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4901e0a-2f07-4b44-81c9-8d029381163d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:47.263187 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c4901e0a-2f07-4b44-81c9-8d029381163d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:47.277391 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4901e0a-2f07-4b44-81c9-8d029381163d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:47.279056 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4901e0a-2f07-4b44-81c9-8d029381163d service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:47.302626 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 902/1802] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:26:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:26:47.406708 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d5be27-7bd3-403a-b859-edaa6a0ee827 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:47.408535 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-09d5be27-7bd3-403a-b859-edaa6a0ee827 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:47.433393 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d5be27-7bd3-403a-b859-edaa6a0ee827 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:47.447190 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bde74ec-903a-4c29-bbeb-414330d7b1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:47.447915 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde74ec-903a-4c29-bbeb-414330d7b1bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:47.449118 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde74ec-903a-4c29-bbeb-414330d7b1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:47.449426 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde74ec-903a-4c29-bbeb-414330d7b1bf admin admin] PATH_INFO: `/v3/projects/5e4fe6f3cee8430da84f3bcbe6fdddb9` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:47.452090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bde74ec-903a-4c29-bbeb-414330d7b1bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e4fe6f3cee8430da84f3bcbe6fdddb9)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:47.456152 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bde74ec-903a-4c29-bbeb-414330d7b1bf admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:47.488550 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:47.583431 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5bde74ec-903a-4c29-bbeb-414330d7b1bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e4fe6f3cee8430da84f3bcbe6fdddb9', 'request_id': 'req-5bde74ec-903a-4c29-bbeb-414330d7b1bf'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:47.609065 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5bde74ec-903a-4c29-bbeb-414330d7b1bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e4fe6f3cee8430da84f3bcbe6fdddb9', 'request_id': 'req-5bde74ec-903a-4c29-bbeb-414330d7b1bf'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:47.619639 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5bde74ec-903a-4c29-bbeb-414330d7b1bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e4fe6f3cee8430da84f3bcbe6fdddb9', 'request_id': 'req-5bde74ec-903a-4c29-bbeb-414330d7b1bf'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:47.632949 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5bde74ec-903a-4c29-bbeb-414330d7b1bf admin admin] The token cache is being invalidate because project 5e4fe6f3cee8430da84f3bcbe6fdddb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:26:47.633313 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5bde74ec-903a-4c29-bbeb-414330d7b1bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:47.637106 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5bde74ec-903a-4c29-bbeb-414330d7b1bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:47.643555 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 901/1803] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:26:47 2022] DELETE /identity/v3/projects/5e4fe6f3cee8430da84f3bcbe6fdddb9 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:47.645056 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4901e0a-2f07-4b44-81c9-8d029381163d service nova] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:47.663314 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4901e0a-2f07-4b44-81c9-8d029381163d service nova] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:47.723140 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:47.742849 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773153b8-031e-4713-be92-a82c5d5805f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:47.744351 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773153b8-031e-4713-be92-a82c5d5805f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:47.744351 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773153b8-031e-4713-be92-a82c5d5805f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:47.744351 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773153b8-031e-4713-be92-a82c5d5805f7 admin admin] PATH_INFO: `/v3/users/9b471c86ff3b4c8ca8963b7316cbcc7c` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:47.746148 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773153b8-031e-4713-be92-a82c5d5805f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b471c86ff3b4c8ca8963b7316cbcc7c)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:47.748306 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773153b8-031e-4713-be92-a82c5d5805f7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:47.823571 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-773153b8-031e-4713-be92-a82c5d5805f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b471c86ff3b4c8ca8963b7316cbcc7c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:47.834096 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-773153b8-031e-4713-be92-a82c5d5805f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b471c86ff3b4c8ca8963b7316cbcc7c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:47.838668 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bde74ec-903a-4c29-bbeb-414330d7b1bf admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:47.840801 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5bde74ec-903a-4c29-bbeb-414330d7b1bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:47.847733 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-773153b8-031e-4713-be92-a82c5d5805f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b471c86ff3b4c8ca8963b7316cbcc7c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:47.861948 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bde74ec-903a-4c29-bbeb-414330d7b1bf admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:47.872650 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-773153b8-031e-4713-be92-a82c5d5805f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b471c86ff3b4c8ca8963b7316cbcc7c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:47.872868 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c15e864-b092-4c25-8894-f0680c670c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:47.873541 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-773153b8-031e-4713-be92-a82c5d5805f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b471c86ff3b4c8ca8963b7316cbcc7c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:47.874055 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c15e864-b092-4c25-8894-f0680c670c27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:47.874393 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c15e864-b092-4c25-8894-f0680c670c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:47.874667 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c15e864-b092-4c25-8894-f0680c670c27 admin admin] PATH_INFO: `/v3/projects/ecf2abd1c72c4e59a4567919abb3181d` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:47.877946 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c15e864-b092-4c25-8894-f0680c670c27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecf2abd1c72c4e59a4567919abb3181d)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:47.880268 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c15e864-b092-4c25-8894-f0680c670c27 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:47.891033 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 903/1804] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:26:47 2022] DELETE /identity/v3/users/9b471c86ff3b4c8ca8963b7316cbcc7c => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:47.958655 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9c15e864-b092-4c25-8894-f0680c670c27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecf2abd1c72c4e59a4567919abb3181d', 'request_id': 'req-9c15e864-b092-4c25-8894-f0680c670c27'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:47.967187 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9c15e864-b092-4c25-8894-f0680c670c27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecf2abd1c72c4e59a4567919abb3181d', 'request_id': 'req-9c15e864-b092-4c25-8894-f0680c670c27'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:47.973225 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9c15e864-b092-4c25-8894-f0680c670c27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecf2abd1c72c4e59a4567919abb3181d', 'request_id': 'req-9c15e864-b092-4c25-8894-f0680c670c27'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:47.985283 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9c15e864-b092-4c25-8894-f0680c670c27 admin admin] The token cache is being invalidate because project ecf2abd1c72c4e59a4567919abb3181d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:26:47.985566 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9c15e864-b092-4c25-8894-f0680c670c27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:47.986212 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9c15e864-b092-4c25-8894-f0680c670c27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:47.988073 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 902/1805] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:26:47 2022] DELETE /identity/v3/projects/ecf2abd1c72c4e59a4567919abb3181d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:26:47.991900 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773153b8-031e-4713-be92-a82c5d5805f7 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:47.993339 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-773153b8-031e-4713-be92-a82c5d5805f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:48.075091 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773153b8-031e-4713-be92-a82c5d5805f7 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:48.159295 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8587200-01d0-4fa5-b161-8d238654daf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:48.160389 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8587200-01d0-4fa5-b161-8d238654daf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:48.160751 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8587200-01d0-4fa5-b161-8d238654daf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:48.161097 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8587200-01d0-4fa5-b161-8d238654daf3 admin admin] PATH_INFO: `/v3/users/93b9e102c7964954a2cacaac1ea7440a` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:48.163152 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8587200-01d0-4fa5-b161-8d238654daf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93b9e102c7964954a2cacaac1ea7440a)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:48.165321 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8587200-01d0-4fa5-b161-8d238654daf3 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:48.194443 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:48.209190 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c15e864-b092-4c25-8894-f0680c670c27 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:48.211008 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9c15e864-b092-4c25-8894-f0680c670c27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:48.231675 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c15e864-b092-4c25-8894-f0680c670c27 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:48.232301 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a8587200-01d0-4fa5-b161-8d238654daf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93b9e102c7964954a2cacaac1ea7440a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:48.237549 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e41b32-5ca8-4235-9680-6bd0cc9da662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:48.238298 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e41b32-5ca8-4235-9680-6bd0cc9da662 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:48.238628 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e41b32-5ca8-4235-9680-6bd0cc9da662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:48.243628 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a8587200-01d0-4fa5-b161-8d238654daf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93b9e102c7964954a2cacaac1ea7440a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:48.244302 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a8587200-01d0-4fa5-b161-8d238654daf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93b9e102c7964954a2cacaac1ea7440a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:48.248240 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e41b32-5ca8-4235-9680-6bd0cc9da662 admin admin] PATH_INFO: `/v3/projects/c987d6c50c48457bb2e07546571159f9` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:48.249687 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e41b32-5ca8-4235-9680-6bd0cc9da662 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c987d6c50c48457bb2e07546571159f9)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:48.251914 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e41b32-5ca8-4235-9680-6bd0cc9da662 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:48.260633 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a8587200-01d0-4fa5-b161-8d238654daf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93b9e102c7964954a2cacaac1ea7440a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:48.264256 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a8587200-01d0-4fa5-b161-8d238654daf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93b9e102c7964954a2cacaac1ea7440a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:48.282080 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 904/1806] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:26:47 2022] DELETE /identity/v3/users/93b9e102c7964954a2cacaac1ea7440a => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:48.330200 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-82e41b32-5ca8-4235-9680-6bd0cc9da662 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c987d6c50c48457bb2e07546571159f9', 'request_id': 'req-82e41b32-5ca8-4235-9680-6bd0cc9da662'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:48.337914 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-82e41b32-5ca8-4235-9680-6bd0cc9da662 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c987d6c50c48457bb2e07546571159f9', 'request_id': 'req-82e41b32-5ca8-4235-9680-6bd0cc9da662'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:48.343904 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-82e41b32-5ca8-4235-9680-6bd0cc9da662 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c987d6c50c48457bb2e07546571159f9', 'request_id': 'req-82e41b32-5ca8-4235-9680-6bd0cc9da662'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:48.355468 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-82e41b32-5ca8-4235-9680-6bd0cc9da662 admin admin] The token cache is being invalidate because project c987d6c50c48457bb2e07546571159f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:26:48.355859 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-82e41b32-5ca8-4235-9680-6bd0cc9da662 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:48.356575 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-82e41b32-5ca8-4235-9680-6bd0cc9da662 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:48.358185 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 903/1807] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:26:48 2022] DELETE /identity/v3/projects/c987d6c50c48457bb2e07546571159f9 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:26:48.365292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d94c56-ab4c-46bd-9bc7-7c33d1db1a7e None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:48.365487 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d94c56-ab4c-46bd-9bc7-7c33d1db1a7e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:48.365783 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d94c56-ab4c-46bd-9bc7-7c33d1db1a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:48.387317 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-e1d94c56-ab4c-46bd-9bc7-7c33d1db1a7e None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:48.388529 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e1d94c56-ab4c-46bd-9bc7-7c33d1db1a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:48.461881 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 905/1808] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:48.470443 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb1a61-b554-4b5e-b785-fdd153cb7c2c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:48.470703 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb1a61-b554-4b5e-b785-fdd153cb7c2c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:48.471243 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb1a61-b554-4b5e-b785-fdd153cb7c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:48.488978 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=76,keystone:DELETE=21,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:48.502015 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9ecb1a61-b554-4b5e-b785-fdd153cb7c2c None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:48.506267 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9ecb1a61-b554-4b5e-b785-fdd153cb7c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:48.515697 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 904/1809] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:26:48.520044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d94c56-ab4c-46bd-9bc7-7c33d1db1a7e None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:48.530525 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d94c56-ab4c-46bd-9bc7-7c33d1db1a7e None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:48.536740 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbee5dc-6926-49c0-8d5c-4447c0aae52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:48.537421 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbee5dc-6926-49c0-8d5c-4447c0aae52a admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:48.537662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbee5dc-6926-49c0-8d5c-4447c0aae52a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:48.537889 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbee5dc-6926-49c0-8d5c-4447c0aae52a admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:48.538639 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbee5dc-6926-49c0-8d5c-4447c0aae52a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:26:48.538888 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbee5dc-6926-49c0-8d5c-4447c0aae52a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:48.540615 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbee5dc-6926-49c0-8d5c-4447c0aae52a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:48.554713 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 906/1810] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:26:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:26:48.558707 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb1a61-b554-4b5e-b785-fdd153cb7c2c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:48.568595 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb1a61-b554-4b5e-b785-fdd153cb7c2c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:48.574481 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c8e5d-7da6-4aa5-b032-d0c66c619ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:48.575264 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c8e5d-7da6-4aa5-b032-d0c66c619ebc admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:48.575630 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c8e5d-7da6-4aa5-b032-d0c66c619ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:48.575985 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c8e5d-7da6-4aa5-b032-d0c66c619ebc admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:48.576853 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c8e5d-7da6-4aa5-b032-d0c66c619ebc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:48.579121 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c8e5d-7da6-4aa5-b032-d0c66c619ebc admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:48.613729 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 905/1811] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:26:48 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:26:48.615481 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbee5dc-6926-49c0-8d5c-4447c0aae52a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:48.627055 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbee5dc-6926-49c0-8d5c-4447c0aae52a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:48.711786 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b87c1c-e65a-40a8-a057-241e1cfdbe30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:48.712780 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b87c1c-e65a-40a8-a057-241e1cfdbe30 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:48.713093 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b87c1c-e65a-40a8-a057-241e1cfdbe30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:48.713630 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b87c1c-e65a-40a8-a057-241e1cfdbe30 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:48.714402 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b87c1c-e65a-40a8-a057-241e1cfdbe30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:48.716793 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b87c1c-e65a-40a8-a057-241e1cfdbe30 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:48.758984 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 907/1812] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:26:48 2022] POST /identity/v3/users => generated 293 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:26:48.763264 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c8e5d-7da6-4aa5-b032-d0c66c619ebc admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:48.774382 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c8e5d-7da6-4aa5-b032-d0c66c619ebc admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:48.777960 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b87c1c-e65a-40a8-a057-241e1cfdbe30 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:48.780511 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-75b87c1c-e65a-40a8-a057-241e1cfdbe30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:48.790644 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93999039-d248-4535-970f-f5bd0c74ae3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:48.791404 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93999039-d248-4535-970f-f5bd0c74ae3a admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:48.791828 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93999039-d248-4535-970f-f5bd0c74ae3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:48.792097 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93999039-d248-4535-970f-f5bd0c74ae3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:48.792869 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93999039-d248-4535-970f-f5bd0c74ae3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:48.794556 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93999039-d248-4535-970f-f5bd0c74ae3a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:48.803207 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b87c1c-e65a-40a8-a057-241e1cfdbe30 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:48.812668 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 906/1813] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:48 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:48.816981 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93999039-d248-4535-970f-f5bd0c74ae3a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:48.819584 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17cad53-24cd-4b64-b31c-cbbb797c2f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:48.820484 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17cad53-24cd-4b64-b31c-cbbb797c2f33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:48.820908 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17cad53-24cd-4b64-b31c-cbbb797c2f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:48.821323 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17cad53-24cd-4b64-b31c-cbbb797c2f33 admin admin] PATH_INFO: `/v3/projects/0a921238708249f9865c942fdd1182d4` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:48.822895 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17cad53-24cd-4b64-b31c-cbbb797c2f33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a921238708249f9865c942fdd1182d4)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:48.826055 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17cad53-24cd-4b64-b31c-cbbb797c2f33 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:48.828228 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93999039-d248-4535-970f-f5bd0c74ae3a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:48.830248 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:48.834326 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f76bb30-a3d9-4571-a382-cdd2f97d7cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:48.834945 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76bb30-a3d9-4571-a382-cdd2f97d7cd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:48.835243 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76bb30-a3d9-4571-a382-cdd2f97d7cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:48.835519 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76bb30-a3d9-4571-a382-cdd2f97d7cd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:48.836218 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f76bb30-a3d9-4571-a382-cdd2f97d7cd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:48.837870 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f76bb30-a3d9-4571-a382-cdd2f97d7cd7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:48.849625 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 907/1814] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:48.853909 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f76bb30-a3d9-4571-a382-cdd2f97d7cd7 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:48.869701 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f76bb30-a3d9-4571-a382-cdd2f97d7cd7 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:48.872063 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d6af78-bfc2-4bbe-bc07-4b1d21c2d3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:48.872776 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6af78-bfc2-4bbe-bc07-4b1d21c2d3ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:48.875214 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6af78-bfc2-4bbe-bc07-4b1d21c2d3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:48.876652 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6af78-bfc2-4bbe-bc07-4b1d21c2d3ce admin admin] PATH_INFO: `/v3/projects/87d0ad2289bc4df88b568265e3ed8905/users/665c39df8a9649dd88eda36d2f16069e/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:48.895835 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d6af78-bfc2-4bbe-bc07-4b1d21c2d3ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=87d0ad2289bc4df88b568265e3ed8905, user_id=665c39df8a9649dd88eda36d2f16069e, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:48.898690 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d6af78-bfc2-4bbe-bc07-4b1d21c2d3ce admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:48.911821 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a17cad53-24cd-4b64-b31c-cbbb797c2f33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a921238708249f9865c942fdd1182d4', 'request_id': 'req-a17cad53-24cd-4b64-b31c-cbbb797c2f33'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:48.920407 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 908/1815] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:26:48 2022] PUT /identity/v3/projects/87d0ad2289bc4df88b568265e3ed8905/users/665c39df8a9649dd88eda36d2f16069e/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:48.920656 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a17cad53-24cd-4b64-b31c-cbbb797c2f33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a921238708249f9865c942fdd1182d4', 'request_id': 'req-a17cad53-24cd-4b64-b31c-cbbb797c2f33'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:48.928824 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a17cad53-24cd-4b64-b31c-cbbb797c2f33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a921238708249f9865c942fdd1182d4', 'request_id': 'req-a17cad53-24cd-4b64-b31c-cbbb797c2f33'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:48.930184 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2f9a4-e6e6-467c-b4b8-66373ea5a74c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:48.930471 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2f9a4-e6e6-467c-b4b8-66373ea5a74c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:48.930752 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2f9a4-e6e6-467c-b4b8-66373ea5a74c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:48.941665 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a17cad53-24cd-4b64-b31c-cbbb797c2f33 admin admin] The token cache is being invalidate because project 0a921238708249f9865c942fdd1182d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:26:48.941954 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a17cad53-24cd-4b64-b31c-cbbb797c2f33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:48.942607 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a17cad53-24cd-4b64-b31c-cbbb797c2f33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:48.947000 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 908/1816] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:26:48 2022] DELETE /identity/v3/projects/0a921238708249f9865c942fdd1182d4 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:48.949207 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc75357-d2bf-4cef-b727-569f9984d0aa None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:48.949560 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc75357-d2bf-4cef-b727-569f9984d0aa None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:48.949833 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc75357-d2bf-4cef-b727-569f9984d0aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:48.971991 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9ea2f9a4-e6e6-467c-b4b8-66373ea5a74c None None] MFA Rules not processed for user `665c39df8a9649dd88eda36d2f16069e`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:48.973554 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9ea2f9a4-e6e6-467c-b4b8-66373ea5a74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:48.976751 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-1cc75357-d2bf-4cef-b727-569f9984d0aa None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:48.978040 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1cc75357-d2bf-4cef-b727-569f9984d0aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:49.062579 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 909/1817] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:26:49.064805 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e714c1-916a-40c8-9731-2a28086ca805 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:49.065264 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e714c1-916a-40c8-9731-2a28086ca805 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:49.065721 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e714c1-916a-40c8-9731-2a28086ca805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:49.072106 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 909/1818] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:26:49.073157 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea2f9a4-e6e6-467c-b4b8-66373ea5a74c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:49.074788 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9ea2f9a4-e6e6-467c-b4b8-66373ea5a74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:49.092765 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea2f9a4-e6e6-467c-b4b8-66373ea5a74c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:49.098162 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb925bb0-6f86-455e-b0c5-84564f5ac56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:49.098998 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb925bb0-6f86-455e-b0c5-84564f5ac56c admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:49.099498 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb925bb0-6f86-455e-b0c5-84564f5ac56c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:49.100362 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb925bb0-6f86-455e-b0c5-84564f5ac56c admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:49.100535 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb925bb0-6f86-455e-b0c5-84564f5ac56c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:49.102943 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb925bb0-6f86-455e-b0c5-84564f5ac56c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:49.103948 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-54e714c1-916a-40c8-9731-2a28086ca805 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:49.105363 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-54e714c1-916a-40c8-9731-2a28086ca805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:49.123133 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 910/1819] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:49.124425 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e714c1-916a-40c8-9731-2a28086ca805 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:49.136323 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e714c1-916a-40c8-9731-2a28086ca805 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:49.140032 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 910/1820] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:26:49 2022] POST /identity/v3/projects => generated 383 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:26:49.142779 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a52e74a-0616-4d71-aa79-9ddfddb75ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:49.143780 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a52e74a-0616-4d71-aa79-9ddfddb75ca6 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:49.144110 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a52e74a-0616-4d71-aa79-9ddfddb75ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:49.144390 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a52e74a-0616-4d71-aa79-9ddfddb75ca6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:49.145050 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb925bb0-6f86-455e-b0c5-84564f5ac56c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:49.145191 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a52e74a-0616-4d71-aa79-9ddfddb75ca6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:26:49.145393 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a52e74a-0616-4d71-aa79-9ddfddb75ca6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:49.147401 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a52e74a-0616-4d71-aa79-9ddfddb75ca6 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:49.175649 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 911/1821] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:26:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:26:49.181814 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a52e74a-0616-4d71-aa79-9ddfddb75ca6 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:49.201294 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a52e74a-0616-4d71-aa79-9ddfddb75ca6 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:49.248940 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb925bb0-6f86-455e-b0c5-84564f5ac56c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:49.254276 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ecaee7-5d21-4c94-91e5-c725b8ff6d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:49.254931 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ecaee7-5d21-4c94-91e5-c725b8ff6d36 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:49.255232 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ecaee7-5d21-4c94-91e5-c725b8ff6d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:49.255510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ecaee7-5d21-4c94-91e5-c725b8ff6d36 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:49.256191 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ecaee7-5d21-4c94-91e5-c725b8ff6d36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:49.258211 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ecaee7-5d21-4c94-91e5-c725b8ff6d36 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:49.267977 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:49.301900 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 911/1822] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:26:49 2022] POST /identity/v3/users => generated 298 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:26:49.302958 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ecaee7-5d21-4c94-91e5-c725b8ff6d36 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:49.329552 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ecaee7-5d21-4c94-91e5-c725b8ff6d36 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:49.338757 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b381b93-cd77-408d-a9a2-1e2b95751e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:49.339122 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c181446-7f5d-4695-990e-a3afd4774468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:49.340450 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b381b93-cd77-408d-a9a2-1e2b95751e6b admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:49.340562 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c181446-7f5d-4695-990e-a3afd4774468 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:49.340647 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b381b93-cd77-408d-a9a2-1e2b95751e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:49.340735 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c181446-7f5d-4695-990e-a3afd4774468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:49.340821 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b381b93-cd77-408d-a9a2-1e2b95751e6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:49.340908 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c181446-7f5d-4695-990e-a3afd4774468 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:49.341316 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c181446-7f5d-4695-990e-a3afd4774468 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:49.341879 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b381b93-cd77-408d-a9a2-1e2b95751e6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:49.343410 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c181446-7f5d-4695-990e-a3afd4774468 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:49.346610 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b381b93-cd77-408d-a9a2-1e2b95751e6b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:49.365886 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 912/1823] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:49.370486 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c181446-7f5d-4695-990e-a3afd4774468 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:49.385584 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c181446-7f5d-4695-990e-a3afd4774468 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:49.392294 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e56d54b-9c3b-49b4-964d-319873442a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:49.394054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56d54b-9c3b-49b4-964d-319873442a0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:49.394441 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56d54b-9c3b-49b4-964d-319873442a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:49.394878 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56d54b-9c3b-49b4-964d-319873442a0a admin admin] PATH_INFO: `/v3/projects/5e0f3a5fc6eb42a5b93c3a745e14d6d1/users/4ef3eeb784cd4d81a61d0def9de98c3b/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:49.410520 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:49.421621 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e56d54b-9c3b-49b4-964d-319873442a0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e0f3a5fc6eb42a5b93c3a745e14d6d1, user_id=4ef3eeb784cd4d81a61d0def9de98c3b, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:49.425231 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 912/1824] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:26:49 2022] POST /identity/v3/projects => generated 381 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:26:49.426499 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e56d54b-9c3b-49b4-964d-319873442a0a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:49.429124 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b381b93-cd77-408d-a9a2-1e2b95751e6b admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:49.456025 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 913/1825] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:26:49 2022] PUT /identity/v3/projects/5e0f3a5fc6eb42a5b93c3a745e14d6d1/users/4ef3eeb784cd4d81a61d0def9de98c3b/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:49.457793 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e56d54b-9c3b-49b4-964d-319873442a0a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:49.513775 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b381b93-cd77-408d-a9a2-1e2b95751e6b admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:49.551771 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e56d54b-9c3b-49b4-964d-319873442a0a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:49.557596 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98c875-2475-46ff-9008-22dbb107d0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:49.558581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98c875-2475-46ff-9008-22dbb107d0b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:49.559018 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98c875-2475-46ff-9008-22dbb107d0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:49.559505 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98c875-2475-46ff-9008-22dbb107d0b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:49.560351 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d98c875-2475-46ff-9008-22dbb107d0b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:49.562132 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d98c875-2475-46ff-9008-22dbb107d0b9 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:49.579042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 914/1826] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:49.583314 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98c875-2475-46ff-9008-22dbb107d0b9 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:49.584910 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1439f6a3-1e0c-4e46-9706-ec33a5e62954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:49.585826 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1439f6a3-1e0c-4e46-9706-ec33a5e62954 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:49.586228 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1439f6a3-1e0c-4e46-9706-ec33a5e62954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:49.586619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1439f6a3-1e0c-4e46-9706-ec33a5e62954 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:49.587403 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1439f6a3-1e0c-4e46-9706-ec33a5e62954 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:49.589431 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1439f6a3-1e0c-4e46-9706-ec33a5e62954 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:49.605633 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98c875-2475-46ff-9008-22dbb107d0b9 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:49.615319 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71c41f4-dfcf-477d-8973-4317d519353a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:49.616288 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c41f4-dfcf-477d-8973-4317d519353a admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:49.616991 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c41f4-dfcf-477d-8973-4317d519353a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:49.617343 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c41f4-dfcf-477d-8973-4317d519353a admin admin] PATH_INFO: `/v3/projects/5e0f3a5fc6eb42a5b93c3a745e14d6d1/users/4ef3eeb784cd4d81a61d0def9de98c3b/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:49.619593 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71c41f4-dfcf-477d-8973-4317d519353a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e0f3a5fc6eb42a5b93c3a745e14d6d1, user_id=4ef3eeb784cd4d81a61d0def9de98c3b, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:49.620026 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 913/1827] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:26:49 2022] POST /identity/v3/users => generated 291 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:26:49.623984 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1439f6a3-1e0c-4e46-9706-ec33a5e62954 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:49.626127 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71c41f4-dfcf-477d-8973-4317d519353a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:49.638999 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 915/1828] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:26:49 2022] PUT /identity/v3/projects/5e0f3a5fc6eb42a5b93c3a745e14d6d1/users/4ef3eeb784cd4d81a61d0def9de98c3b/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:49.639142 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1439f6a3-1e0c-4e46-9706-ec33a5e62954 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:49.649378 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36c1ae-9fa9-455a-a276-d84ae1261169 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:49.649893 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36c1ae-9fa9-455a-a276-d84ae1261169 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:49.650167 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36c1ae-9fa9-455a-a276-d84ae1261169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:49.701944 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-be36c1ae-9fa9-455a-a276-d84ae1261169 None None] MFA Rules not processed for user `4ef3eeb784cd4d81a61d0def9de98c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:49.703970 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-be36c1ae-9fa9-455a-a276-d84ae1261169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:49.727972 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d24dd34-0c62-498d-a8d4-ad0fa919318b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:49.728481 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24dd34-0c62-498d-a8d4-ad0fa919318b admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:49.728761 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24dd34-0c62-498d-a8d4-ad0fa919318b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:49.729165 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24dd34-0c62-498d-a8d4-ad0fa919318b admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:49.729752 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d24dd34-0c62-498d-a8d4-ad0fa919318b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:49.731790 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d24dd34-0c62-498d-a8d4-ad0fa919318b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:49.746287 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 914/1829] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:49.747573 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d24dd34-0c62-498d-a8d4-ad0fa919318b admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:49.759784 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d24dd34-0c62-498d-a8d4-ad0fa919318b admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:49.767213 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accea4ee-fd10-4be6-b025-172ffd0c6e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:49.769654 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accea4ee-fd10-4be6-b025-172ffd0c6e0a admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:49.770173 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accea4ee-fd10-4be6-b025-172ffd0c6e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:49.770754 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accea4ee-fd10-4be6-b025-172ffd0c6e0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:49.772033 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accea4ee-fd10-4be6-b025-172ffd0c6e0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:49.774006 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accea4ee-fd10-4be6-b025-172ffd0c6e0a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:49.794775 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 915/1830] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:49.794994 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 916/1831] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:49 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:49.796324 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accea4ee-fd10-4be6-b025-172ffd0c6e0a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:49.807941 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be36c1ae-9fa9-455a-a276-d84ae1261169 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:49.808164 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accea4ee-fd10-4be6-b025-172ffd0c6e0a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:49.810120 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-be36c1ae-9fa9-455a-a276-d84ae1261169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:49.814443 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebf49bf-f1c2-4c3c-8f06-291683c99d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:49.815437 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf49bf-f1c2-4c3c-8f06-291683c99d26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:49.815731 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf49bf-f1c2-4c3c-8f06-291683c99d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:49.816007 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf49bf-f1c2-4c3c-8f06-291683c99d26 admin admin] PATH_INFO: `/v3/projects/50610758ecc2421abe60b6be7f486f32/users/7056c268898c47d48438ab5eaab1c8e2/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:49.836872 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebf49bf-f1c2-4c3c-8f06-291683c99d26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50610758ecc2421abe60b6be7f486f32, user_id=7056c268898c47d48438ab5eaab1c8e2, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:49.839987 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebf49bf-f1c2-4c3c-8f06-291683c99d26 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:49.845750 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:49.847883 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:49.873623 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 916/1832] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:26:49 2022] PUT /identity/v3/projects/50610758ecc2421abe60b6be7f486f32/users/7056c268898c47d48438ab5eaab1c8e2/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:26:49.885735 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57685d7f-9772-4c4c-a26f-38419dc86489 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:49.886163 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57685d7f-9772-4c4c-a26f-38419dc86489 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:49.886498 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57685d7f-9772-4c4c-a26f-38419dc86489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:49.905242 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be36c1ae-9fa9-455a-a276-d84ae1261169 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:49.926165 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-57685d7f-9772-4c4c-a26f-38419dc86489 None None] MFA Rules not processed for user `7056c268898c47d48438ab5eaab1c8e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:49.927547 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-57685d7f-9772-4c4c-a26f-38419dc86489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:49.972607 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f48241-8b25-4661-b4aa-ff4d5cb7702a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:49.973591 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f48241-8b25-4661-b4aa-ff4d5cb7702a service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:49.973871 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f48241-8b25-4661-b4aa-ff4d5cb7702a service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:49.974140 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f48241-8b25-4661-b4aa-ff4d5cb7702a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:50.039211 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 917/1833] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:50.039501 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f48241-8b25-4661-b4aa-ff4d5cb7702a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:50.040830 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57685d7f-9772-4c4c-a26f-38419dc86489 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:50.040950 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f48241-8b25-4661-b4aa-ff4d5cb7702a service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:50.055094 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 917/1834] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:26:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:26:50.118162 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57685d7f-9772-4c4c-a26f-38419dc86489 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:50.123446 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e9e596-ca32-4a65-bf29-87515421d650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:50.124340 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9e596-ca32-4a65-bf29-87515421d650 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:50.124861 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9e596-ca32-4a65-bf29-87515421d650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:50.125152 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9e596-ca32-4a65-bf29-87515421d650 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:50.126124 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e9e596-ca32-4a65-bf29-87515421d650 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:50.128165 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e9e596-ca32-4a65-bf29-87515421d650 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:50.186809 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 918/1835] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:26:50 2022] POST /identity/v3/projects => generated 381 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:26:50.187888 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f48241-8b25-4661-b4aa-ff4d5cb7702a service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:50.202732 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f48241-8b25-4661-b4aa-ff4d5cb7702a service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:50.290537 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b62f5-922a-4595-9979-2d990d00dcc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:50.291310 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b62f5-922a-4595-9979-2d990d00dcc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:50.292312 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b62f5-922a-4595-9979-2d990d00dcc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:50.292312 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b62f5-922a-4595-9979-2d990d00dcc9 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:50.292719 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591b62f5-922a-4595-9979-2d990d00dcc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:50.295022 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591b62f5-922a-4595-9979-2d990d00dcc9 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:50.322528 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 918/1836] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:26:50 2022] POST /identity/v3/users => generated 297 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:26:50.326641 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e9e596-ca32-4a65-bf29-87515421d650 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:50.339684 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e9e596-ca32-4a65-bf29-87515421d650 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:50.345021 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb88b8-ce8a-45c8-bc06-18c2ae8e945d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:50.345654 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb88b8-ce8a-45c8-bc06-18c2ae8e945d admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:50.346169 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb88b8-ce8a-45c8-bc06-18c2ae8e945d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:50.346169 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb88b8-ce8a-45c8-bc06-18c2ae8e945d admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:50.346973 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bb88b8-ce8a-45c8-bc06-18c2ae8e945d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:50.349110 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bb88b8-ce8a-45c8-bc06-18c2ae8e945d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:50.362504 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 919/1837] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:50.366792 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b62f5-922a-4595-9979-2d990d00dcc9 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:50.380772 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b62f5-922a-4595-9979-2d990d00dcc9 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:50.386229 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d0c552-2143-4fcc-91ec-c1a0bf71d758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:50.386996 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d0c552-2143-4fcc-91ec-c1a0bf71d758 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:50.387254 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d0c552-2143-4fcc-91ec-c1a0bf71d758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:50.387515 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d0c552-2143-4fcc-91ec-c1a0bf71d758 admin admin] PATH_INFO: `/v3/projects/c210e88f9f89439bb5d037e73f469331/users/76a5b6ab178d422a8a64ca7e574b4ed4/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:50.406639 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d0c552-2143-4fcc-91ec-c1a0bf71d758 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c210e88f9f89439bb5d037e73f469331, user_id=76a5b6ab178d422a8a64ca7e574b4ed4, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:50.409990 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d0c552-2143-4fcc-91ec-c1a0bf71d758 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:50.421557 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:50.434842 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 919/1838] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:26:50 2022] PUT /identity/v3/projects/c210e88f9f89439bb5d037e73f469331/users/76a5b6ab178d422a8a64ca7e574b4ed4/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:26:50.439553 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb88b8-ce8a-45c8-bc06-18c2ae8e945d admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:50.465362 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb88b8-ce8a-45c8-bc06-18c2ae8e945d admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:50.556282 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429cf305-2208-460e-87e4-b26113fde825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:50.557250 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429cf305-2208-460e-87e4-b26113fde825 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:50.558400 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429cf305-2208-460e-87e4-b26113fde825 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:50.559232 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429cf305-2208-460e-87e4-b26113fde825 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:50.560279 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429cf305-2208-460e-87e4-b26113fde825 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:50.562513 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429cf305-2208-460e-87e4-b26113fde825 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:50.565162 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:50.579140 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 920/1839] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:50.582932 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d0c552-2143-4fcc-91ec-c1a0bf71d758 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:50.596009 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d0c552-2143-4fcc-91ec-c1a0bf71d758 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:50.601769 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f4c11-895a-489c-b56c-f7042cd3f92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:50.602801 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f4c11-895a-489c-b56c-f7042cd3f92b admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:50.603317 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f4c11-895a-489c-b56c-f7042cd3f92b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:50.603918 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f4c11-895a-489c-b56c-f7042cd3f92b admin admin] PATH_INFO: `/v3/projects/c210e88f9f89439bb5d037e73f469331/users/76a5b6ab178d422a8a64ca7e574b4ed4/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:50.606071 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446f4c11-895a-489c-b56c-f7042cd3f92b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c210e88f9f89439bb5d037e73f469331, user_id=76a5b6ab178d422a8a64ca7e574b4ed4, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:50.608921 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446f4c11-895a-489c-b56c-f7042cd3f92b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:50.624254 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 920/1840] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:26:50 2022] PUT /identity/v3/projects/c210e88f9f89439bb5d037e73f469331/users/76a5b6ab178d422a8a64ca7e574b4ed4/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:50.634783 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5f52d-2780-4621-9a45-6f96917448c7 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:50.635136 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5f52d-2780-4621-9a45-6f96917448c7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:50.635424 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5f52d-2780-4621-9a45-6f96917448c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:50.669740 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-9df5f52d-2780-4621-9a45-6f96917448c7 None None] MFA Rules not processed for user `76a5b6ab178d422a8a64ca7e574b4ed4`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:50.670976 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9df5f52d-2780-4621-9a45-6f96917448c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:50.743002 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 921/1841] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:50 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:50.754495 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f4c11-895a-489c-b56c-f7042cd3f92b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:50.811395 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f4c11-895a-489c-b56c-f7042cd3f92b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:50.815355 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b614f011-85ee-45ff-aa51-03e859548ecc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:50.815997 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b614f011-85ee-45ff-aa51-03e859548ecc service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:50.816286 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b614f011-85ee-45ff-aa51-03e859548ecc service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:50.816564 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b614f011-85ee-45ff-aa51-03e859548ecc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:50.827236 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b614f011-85ee-45ff-aa51-03e859548ecc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:50.828917 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b614f011-85ee-45ff-aa51-03e859548ecc service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:50.839107 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 921/1842] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:26:50 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:26:50.944183 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df5f52d-2780-4621-9a45-6f96917448c7 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:50.954550 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df5f52d-2780-4621-9a45-6f96917448c7 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:50.958510 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7093a19-02a4-43bf-84b4-eaaa7e5ddf72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:50.959172 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7093a19-02a4-43bf-84b4-eaaa7e5ddf72 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:50.959457 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7093a19-02a4-43bf-84b4-eaaa7e5ddf72 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:50.959726 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7093a19-02a4-43bf-84b4-eaaa7e5ddf72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:51.016558 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7093a19-02a4-43bf-84b4-eaaa7e5ddf72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:51.017995 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7093a19-02a4-43bf-84b4-eaaa7e5ddf72 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:51.027085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 922/1843] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:26:50 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:26:53.707368 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b614f011-85ee-45ff-aa51-03e859548ecc service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:53.775552 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b614f011-85ee-45ff-aa51-03e859548ecc service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:53.784232 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ace63b8-e8b7-4e42-b887-f211ac23eeb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:53.785008 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace63b8-e8b7-4e42-b887-f211ac23eeb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:53.785365 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace63b8-e8b7-4e42-b887-f211ac23eeb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:53.785712 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace63b8-e8b7-4e42-b887-f211ac23eeb8 admin admin] PATH_INFO: `/v3/users/7056c268898c47d48438ab5eaab1c8e2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:53.787426 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ace63b8-e8b7-4e42-b887-f211ac23eeb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7056c268898c47d48438ab5eaab1c8e2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:53.789582 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ace63b8-e8b7-4e42-b887-f211ac23eeb8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:53.858294 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2ace63b8-e8b7-4e42-b887-f211ac23eeb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7056c268898c47d48438ab5eaab1c8e2'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:53.858981 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2ace63b8-e8b7-4e42-b887-f211ac23eeb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7056c268898c47d48438ab5eaab1c8e2'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:53.859585 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2ace63b8-e8b7-4e42-b887-f211ac23eeb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7056c268898c47d48438ab5eaab1c8e2'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:53.867129 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2ace63b8-e8b7-4e42-b887-f211ac23eeb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7056c268898c47d48438ab5eaab1c8e2'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:53.874494 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2ace63b8-e8b7-4e42-b887-f211ac23eeb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7056c268898c47d48438ab5eaab1c8e2'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:53.895537 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 922/1844] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:26:53 2022] DELETE /identity/v3/users/7056c268898c47d48438ab5eaab1c8e2 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:53.900079 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5bb8d-5a76-445c-8ce5-adf15697fb68 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:53.900605 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5bb8d-5a76-445c-8ce5-adf15697fb68 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:53.900605 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5bb8d-5a76-445c-8ce5-adf15697fb68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:53.920191 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-e8e5bb8d-5a76-445c-8ce5-adf15697fb68 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:53.921392 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e8e5bb8d-5a76-445c-8ce5-adf15697fb68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:53.983084 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 923/1845] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:53.992827 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ace63b8-e8b7-4e42-b887-f211ac23eeb8 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:53.994244 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2ace63b8-e8b7-4e42-b887-f211ac23eeb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:54.004124 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:54.057225 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ace63b8-e8b7-4e42-b887-f211ac23eeb8 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:54.060979 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e605f-79de-45e3-894b-64a56c700f28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:54.061593 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e605f-79de-45e3-894b-64a56c700f28 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:54.061874 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e605f-79de-45e3-894b-64a56c700f28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:54.062145 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e605f-79de-45e3-894b-64a56c700f28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:54.071586 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5e605f-79de-45e3-894b-64a56c700f28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:54.073038 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5e605f-79de-45e3-894b-64a56c700f28 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:54.082728 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 923/1846] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:26:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:26:54.459735 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e5bb8d-5a76-445c-8ce5-adf15697fb68 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:54.461207 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e8e5bb8d-5a76-445c-8ce5-adf15697fb68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:54.472512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e605f-79de-45e3-894b-64a56c700f28 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:54.474622 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3f5e605f-79de-45e3-894b-64a56c700f28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:54.486080 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:54.487583 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e605f-79de-45e3-894b-64a56c700f28 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:54.493817 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd400717-eb26-44a4-bf98-68e35391f9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:54.494498 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd400717-eb26-44a4-bf98-68e35391f9e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:54.494935 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd400717-eb26-44a4-bf98-68e35391f9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:54.495136 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd400717-eb26-44a4-bf98-68e35391f9e5 admin admin] PATH_INFO: `/v3/projects/50610758ecc2421abe60b6be7f486f32` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:54.496837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd400717-eb26-44a4-bf98-68e35391f9e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50610758ecc2421abe60b6be7f486f32)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:54.498743 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd400717-eb26-44a4-bf98-68e35391f9e5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:54.535087 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e5bb8d-5a76-445c-8ce5-adf15697fb68 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:54.539689 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025eda1-478f-4d45-a159-3103f7825c50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:54.540353 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025eda1-478f-4d45-a159-3103f7825c50 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:54.540630 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025eda1-478f-4d45-a159-3103f7825c50 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:54.541317 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025eda1-478f-4d45-a159-3103f7825c50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:54.543325 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1025eda1-478f-4d45-a159-3103f7825c50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:54.567595 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-bd400717-eb26-44a4-bf98-68e35391f9e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50610758ecc2421abe60b6be7f486f32', 'request_id': 'req-bd400717-eb26-44a4-bf98-68e35391f9e5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:54.576853 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-bd400717-eb26-44a4-bf98-68e35391f9e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50610758ecc2421abe60b6be7f486f32', 'request_id': 'req-bd400717-eb26-44a4-bf98-68e35391f9e5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:54.582982 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-bd400717-eb26-44a4-bf98-68e35391f9e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50610758ecc2421abe60b6be7f486f32', 'request_id': 'req-bd400717-eb26-44a4-bf98-68e35391f9e5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:54.595417 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-bd400717-eb26-44a4-bf98-68e35391f9e5 admin admin] The token cache is being invalidate because project 50610758ecc2421abe60b6be7f486f32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:26:54.595706 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-bd400717-eb26-44a4-bf98-68e35391f9e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:54.596831 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-bd400717-eb26-44a4-bf98-68e35391f9e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:54.597874 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 924/1847] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:26:54 2022] DELETE /identity/v3/projects/50610758ecc2421abe60b6be7f486f32 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:54.601703 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd400717-eb26-44a4-bf98-68e35391f9e5 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:54.603020 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bd400717-eb26-44a4-bf98-68e35391f9e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:54.612775 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1025eda1-478f-4d45-a159-3103f7825c50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:54.614215 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1025eda1-478f-4d45-a159-3103f7825c50 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:54.623523 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 924/1848] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:26:54 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:26:54.670164 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd400717-eb26-44a4-bf98-68e35391f9e5 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:54.675035 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f5147c-96b0-486b-95af-1508a833961e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:54.675696 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5147c-96b0-486b-95af-1508a833961e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:54.675975 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5147c-96b0-486b-95af-1508a833961e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:54.676244 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5147c-96b0-486b-95af-1508a833961e admin admin] PATH_INFO: `/v3/users/76a5b6ab178d422a8a64ca7e574b4ed4` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:54.677419 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f5147c-96b0-486b-95af-1508a833961e admin admin] RBAC: Authorizing `identity:delete_user(user_id=76a5b6ab178d422a8a64ca7e574b4ed4)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:54.679371 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f5147c-96b0-486b-95af-1508a833961e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:54.724813 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:54.737988 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-74f5147c-96b0-486b-95af-1508a833961e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76a5b6ab178d422a8a64ca7e574b4ed4'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:54.738712 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-74f5147c-96b0-486b-95af-1508a833961e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76a5b6ab178d422a8a64ca7e574b4ed4'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:54.739758 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-74f5147c-96b0-486b-95af-1508a833961e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76a5b6ab178d422a8a64ca7e574b4ed4'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:54.749871 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-74f5147c-96b0-486b-95af-1508a833961e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76a5b6ab178d422a8a64ca7e574b4ed4'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:54.758554 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-74f5147c-96b0-486b-95af-1508a833961e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76a5b6ab178d422a8a64ca7e574b4ed4'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:54.783142 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 925/1849] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:26:54 2022] DELETE /identity/v3/users/76a5b6ab178d422a8a64ca7e574b4ed4 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:26:55.435408 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025eda1-478f-4d45-a159-3103f7825c50 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:55.448083 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025eda1-478f-4d45-a159-3103f7825c50 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:55.524091 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc22c02-49b0-4f03-b3db-ab1e33eb5071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:55.524759 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc22c02-49b0-4f03-b3db-ab1e33eb5071 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:55.525038 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc22c02-49b0-4f03-b3db-ab1e33eb5071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:55.525304 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc22c02-49b0-4f03-b3db-ab1e33eb5071 admin admin] PATH_INFO: `/v3/projects/c210e88f9f89439bb5d037e73f469331` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:55.526447 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc22c02-49b0-4f03-b3db-ab1e33eb5071 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c210e88f9f89439bb5d037e73f469331)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:55.528378 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc22c02-49b0-4f03-b3db-ab1e33eb5071 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:55.578357 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4cc22c02-49b0-4f03-b3db-ab1e33eb5071 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c210e88f9f89439bb5d037e73f469331', 'request_id': 'req-4cc22c02-49b0-4f03-b3db-ab1e33eb5071'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:55.584236 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4cc22c02-49b0-4f03-b3db-ab1e33eb5071 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c210e88f9f89439bb5d037e73f469331', 'request_id': 'req-4cc22c02-49b0-4f03-b3db-ab1e33eb5071'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:55.594073 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4cc22c02-49b0-4f03-b3db-ab1e33eb5071 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c210e88f9f89439bb5d037e73f469331', 'request_id': 'req-4cc22c02-49b0-4f03-b3db-ab1e33eb5071'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:55.604409 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:55.610040 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4cc22c02-49b0-4f03-b3db-ab1e33eb5071 admin admin] The token cache is being invalidate because project c210e88f9f89439bb5d037e73f469331 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:26:55.610457 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4cc22c02-49b0-4f03-b3db-ab1e33eb5071 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:55.612547 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4cc22c02-49b0-4f03-b3db-ab1e33eb5071 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:55.615449 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 925/1850] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:26:55 2022] DELETE /identity/v3/projects/c210e88f9f89439bb5d037e73f469331 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:55.623530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10b6b1e-6da1-4b42-85c7-b6e2602e8863 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:55.623770 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10b6b1e-6da1-4b42-85c7-b6e2602e8863 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:55.623978 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10b6b1e-6da1-4b42-85c7-b6e2602e8863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:55.649647 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b10b6b1e-6da1-4b42-85c7-b6e2602e8863 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:55.650931 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b10b6b1e-6da1-4b42-85c7-b6e2602e8863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:55.726932 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 926/1851] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:55.732200 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27f3c2-b6db-4874-8180-bb54b6d362b7 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:55.732525 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27f3c2-b6db-4874-8180-bb54b6d362b7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:55.732823 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27f3c2-b6db-4874-8180-bb54b6d362b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:55.753937 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-fd27f3c2-b6db-4874-8180-bb54b6d362b7 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:55.755231 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fd27f3c2-b6db-4874-8180-bb54b6d362b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:55.763219 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 926/1852] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:55.768249 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10b6b1e-6da1-4b42-85c7-b6e2602e8863 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:55.783500 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10b6b1e-6da1-4b42-85c7-b6e2602e8863 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:55.791666 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ee163-9e29-44a4-a606-9676d98527b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:55.792478 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ee163-9e29-44a4-a606-9676d98527b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:55.792892 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ee163-9e29-44a4-a606-9676d98527b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:55.793312 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ee163-9e29-44a4-a606-9676d98527b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:55.794345 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ee163-9e29-44a4-a606-9676d98527b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:26:55.794788 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ee163-9e29-44a4-a606-9676d98527b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:55.797321 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ee163-9e29-44a4-a606-9676d98527b4 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:55.810664 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 927/1853] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:26:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:26:55.816942 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd27f3c2-b6db-4874-8180-bb54b6d362b7 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:55.827094 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd27f3c2-b6db-4874-8180-bb54b6d362b7 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:55.832402 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a0d8f3-1476-45da-9ad3-354500380df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:55.833018 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a0d8f3-1476-45da-9ad3-354500380df5 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:55.833291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a0d8f3-1476-45da-9ad3-354500380df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:55.833562 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a0d8f3-1476-45da-9ad3-354500380df5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:55.834229 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a0d8f3-1476-45da-9ad3-354500380df5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:55.836174 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a0d8f3-1476-45da-9ad3-354500380df5 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:55.868830 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 927/1854] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:26:55 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:26:55.869960 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ee163-9e29-44a4-a606-9676d98527b4 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:55.880532 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ee163-9e29-44a4-a606-9676d98527b4 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:55.938810 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb7415c-0fbd-496a-8677-e95086a71890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:55.939598 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb7415c-0fbd-496a-8677-e95086a71890 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:55.939876 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb7415c-0fbd-496a-8677-e95086a71890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:55.940147 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb7415c-0fbd-496a-8677-e95086a71890 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:55.940825 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb7415c-0fbd-496a-8677-e95086a71890 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:55.942739 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb7415c-0fbd-496a-8677-e95086a71890 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:55.973953 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 928/1855] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:26:55 2022] POST /identity/v3/users => generated 294 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:26:55.975419 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a0d8f3-1476-45da-9ad3-354500380df5 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:55.987726 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a0d8f3-1476-45da-9ad3-354500380df5 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:55.995908 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d0dcb8-b5d3-4a0f-a9dc-ea5225269636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:55.996718 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d0dcb8-b5d3-4a0f-a9dc-ea5225269636 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:55.997032 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d0dcb8-b5d3-4a0f-a9dc-ea5225269636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:55.997408 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d0dcb8-b5d3-4a0f-a9dc-ea5225269636 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:55.998178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d0dcb8-b5d3-4a0f-a9dc-ea5225269636 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:56.000326 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d0dcb8-b5d3-4a0f-a9dc-ea5225269636 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:56.016771 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb7415c-0fbd-496a-8677-e95086a71890 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:56.019545 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 928/1856] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:56.027909 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb7415c-0fbd-496a-8677-e95086a71890 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:56.032922 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d656f8cc-e006-4e99-ab41-656cbf793d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:56.033553 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656f8cc-e006-4e99-ab41-656cbf793d38 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:56.033835 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656f8cc-e006-4e99-ab41-656cbf793d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:56.034114 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656f8cc-e006-4e99-ab41-656cbf793d38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:56.034824 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d656f8cc-e006-4e99-ab41-656cbf793d38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:56.036833 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d656f8cc-e006-4e99-ab41-656cbf793d38 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:56.048462 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 929/1857] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:56.052631 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d0dcb8-b5d3-4a0f-a9dc-ea5225269636 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:56.062793 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d0dcb8-b5d3-4a0f-a9dc-ea5225269636 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:56.067670 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a702b76c-0a03-448b-98d2-0107f8f571f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:56.068341 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702b76c-0a03-448b-98d2-0107f8f571f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:56.068625 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702b76c-0a03-448b-98d2-0107f8f571f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:56.068947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702b76c-0a03-448b-98d2-0107f8f571f7 admin admin] PATH_INFO: `/v3/projects/562d0d464a2b4385b8f4809b60789b7c/users/8d30241d6fc149c696b2ec23a860af1a/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:56.083636 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a702b76c-0a03-448b-98d2-0107f8f571f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=562d0d464a2b4385b8f4809b60789b7c, user_id=8d30241d6fc149c696b2ec23a860af1a, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:56.086297 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a702b76c-0a03-448b-98d2-0107f8f571f7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:56.105827 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 929/1858] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:26:56 2022] PUT /identity/v3/projects/562d0d464a2b4385b8f4809b60789b7c/users/8d30241d6fc149c696b2ec23a860af1a/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:56.120506 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cef00-6388-4089-ab38-1e876251fe9e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:56.120787 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cef00-6388-4089-ab38-1e876251fe9e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:56.121147 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cef00-6388-4089-ab38-1e876251fe9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:56.156904 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-0b7cef00-6388-4089-ab38-1e876251fe9e None None] MFA Rules not processed for user `8d30241d6fc149c696b2ec23a860af1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:56.158605 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0b7cef00-6388-4089-ab38-1e876251fe9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:56.165065 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:56.235471 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 930/1859] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:56.240196 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a702b76c-0a03-448b-98d2-0107f8f571f7 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:56.304289 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a702b76c-0a03-448b-98d2-0107f8f571f7 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:56.308987 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d130d45-a4e1-49a7-8802-50c0da9671e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:56.309609 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d130d45-a4e1-49a7-8802-50c0da9671e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:56.309891 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d130d45-a4e1-49a7-8802-50c0da9671e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:56.310167 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d130d45-a4e1-49a7-8802-50c0da9671e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:56.310833 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d130d45-a4e1-49a7-8802-50c0da9671e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:56.312821 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d130d45-a4e1-49a7-8802-50c0da9671e2 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:56.343197 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 930/1860] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:26:56 2022] POST /identity/v3/projects => generated 387 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:26:56.348207 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7cef00-6388-4089-ab38-1e876251fe9e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:56.365721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7cef00-6388-4089-ab38-1e876251fe9e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:56.429452 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c12568-a6c7-445f-8685-2e00fc20b7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:56.430076 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c12568-a6c7-445f-8685-2e00fc20b7d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:56.430355 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c12568-a6c7-445f-8685-2e00fc20b7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:56.430629 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c12568-a6c7-445f-8685-2e00fc20b7d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:56.431344 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c12568-a6c7-445f-8685-2e00fc20b7d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:56.433260 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c12568-a6c7-445f-8685-2e00fc20b7d1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:56.457684 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 931/1861] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:26:56 2022] POST /identity/v3/users => generated 300 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:26:56.461837 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d130d45-a4e1-49a7-8802-50c0da9671e2 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:56.472932 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d130d45-a4e1-49a7-8802-50c0da9671e2 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:56.478348 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa67ce-a6cb-43ba-892f-0d7e8472f612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:56.478993 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa67ce-a6cb-43ba-892f-0d7e8472f612 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:56.479276 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa67ce-a6cb-43ba-892f-0d7e8472f612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:56.479546 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa67ce-a6cb-43ba-892f-0d7e8472f612 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:56.480292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aa67ce-a6cb-43ba-892f-0d7e8472f612 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:56.481923 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aa67ce-a6cb-43ba-892f-0d7e8472f612 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:56.494766 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 931/1862] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:56.499091 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c12568-a6c7-445f-8685-2e00fc20b7d1 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:56.509893 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c12568-a6c7-445f-8685-2e00fc20b7d1 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:56.515393 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d7d4c8-7da6-45b4-9836-cbc0669e4847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:56.516140 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7d4c8-7da6-45b4-9836-cbc0669e4847 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:56.516345 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7d4c8-7da6-45b4-9836-cbc0669e4847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:56.517147 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7d4c8-7da6-45b4-9836-cbc0669e4847 admin admin] PATH_INFO: `/v3/projects/2e7e8cb23de249eb854a4c11eb8269f3/users/4e67c38840a2436bb24b3cecdef1f2f6/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:56.534346 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d7d4c8-7da6-45b4-9836-cbc0669e4847 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e7e8cb23de249eb854a4c11eb8269f3, user_id=4e67c38840a2436bb24b3cecdef1f2f6, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:56.537481 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d7d4c8-7da6-45b4-9836-cbc0669e4847 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:56.557347 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 932/1863] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:26:56 2022] PUT /identity/v3/projects/2e7e8cb23de249eb854a4c11eb8269f3/users/4e67c38840a2436bb24b3cecdef1f2f6/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:56.561095 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa67ce-a6cb-43ba-892f-0d7e8472f612 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:56.572308 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa67ce-a6cb-43ba-892f-0d7e8472f612 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:56.632067 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5967236a-5326-4c9d-87be-6e8ebb65064a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:56.632781 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967236a-5326-4c9d-87be-6e8ebb65064a admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:56.633156 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967236a-5326-4c9d-87be-6e8ebb65064a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:56.633491 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967236a-5326-4c9d-87be-6e8ebb65064a admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:56.634249 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5967236a-5326-4c9d-87be-6e8ebb65064a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:56.635939 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5967236a-5326-4c9d-87be-6e8ebb65064a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:56.647190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 932/1864] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:26:56.651752 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d7d4c8-7da6-45b4-9836-cbc0669e4847 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:56.666798 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d7d4c8-7da6-45b4-9836-cbc0669e4847 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:56.674869 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f865fa-360e-4b4d-86cc-46cef022df2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:56.676286 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f865fa-360e-4b4d-86cc-46cef022df2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:56.676587 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f865fa-360e-4b4d-86cc-46cef022df2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:56.676850 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f865fa-360e-4b4d-86cc-46cef022df2a admin admin] PATH_INFO: `/v3/projects/2e7e8cb23de249eb854a4c11eb8269f3/users/4e67c38840a2436bb24b3cecdef1f2f6/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:56.679221 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f865fa-360e-4b4d-86cc-46cef022df2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e7e8cb23de249eb854a4c11eb8269f3, user_id=4e67c38840a2436bb24b3cecdef1f2f6, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:56.682125 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f865fa-360e-4b4d-86cc-46cef022df2a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:56.694773 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 933/1865] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:26:56 2022] PUT /identity/v3/projects/2e7e8cb23de249eb854a4c11eb8269f3/users/4e67c38840a2436bb24b3cecdef1f2f6/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:56.709813 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5ee33-be4c-4288-87e5-7e7d0a0eadda None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:56.710128 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5ee33-be4c-4288-87e5-7e7d0a0eadda None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:56.710409 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5ee33-be4c-4288-87e5-7e7d0a0eadda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:56.725141 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:56.740463 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-5ab5ee33-be4c-4288-87e5-7e7d0a0eadda None None] MFA Rules not processed for user `4e67c38840a2436bb24b3cecdef1f2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:56.741794 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5ab5ee33-be4c-4288-87e5-7e7d0a0eadda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:56.815579 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 933/1866] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:56 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:56.826361 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f865fa-360e-4b4d-86cc-46cef022df2a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:56.828233 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a0f865fa-360e-4b4d-86cc-46cef022df2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:56.886713 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f865fa-360e-4b4d-86cc-46cef022df2a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:56.890285 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c76dc9-2c88-491b-b5c0-c247596748c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:56.891013 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c76dc9-2c88-491b-b5c0-c247596748c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:56.891396 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c76dc9-2c88-491b-b5c0-c247596748c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:56.891847 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c76dc9-2c88-491b-b5c0-c247596748c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:56.901321 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c76dc9-2c88-491b-b5c0-c247596748c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:56.902753 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c76dc9-2c88-491b-b5c0-c247596748c0 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:56.912590 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 934/1867] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:26:56 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:26:57.539850 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5ee33-be4c-4288-87e5-7e7d0a0eadda None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:57.603639 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5ee33-be4c-4288-87e5-7e7d0a0eadda None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:57.608602 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251a620-80a1-41bb-b705-fb06bdd74c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:57.609372 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251a620-80a1-41bb-b705-fb06bdd74c9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:57.609571 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251a620-80a1-41bb-b705-fb06bdd74c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:57.609827 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251a620-80a1-41bb-b705-fb06bdd74c9a admin admin] PATH_INFO: `/v3/users/8d30241d6fc149c696b2ec23a860af1a` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:57.611296 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5251a620-80a1-41bb-b705-fb06bdd74c9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d30241d6fc149c696b2ec23a860af1a)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:57.613198 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5251a620-80a1-41bb-b705-fb06bdd74c9a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:57.666386 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5251a620-80a1-41bb-b705-fb06bdd74c9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d30241d6fc149c696b2ec23a860af1a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:57.675151 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5251a620-80a1-41bb-b705-fb06bdd74c9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d30241d6fc149c696b2ec23a860af1a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:57.675947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5251a620-80a1-41bb-b705-fb06bdd74c9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d30241d6fc149c696b2ec23a860af1a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:57.696394 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5251a620-80a1-41bb-b705-fb06bdd74c9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d30241d6fc149c696b2ec23a860af1a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:57.697409 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5251a620-80a1-41bb-b705-fb06bdd74c9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d30241d6fc149c696b2ec23a860af1a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:57.711498 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 934/1868] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:26:57 2022] DELETE /identity/v3/users/8d30241d6fc149c696b2ec23a860af1a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:57.716348 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8b0a1-b52c-4eda-b520-c1a8d482cb1a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:57.716635 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8b0a1-b52c-4eda-b520-c1a8d482cb1a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:57.716903 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8b0a1-b52c-4eda-b520-c1a8d482cb1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:57.725283 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:57.739271 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-05f8b0a1-b52c-4eda-b520-c1a8d482cb1a None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:57.740743 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-05f8b0a1-b52c-4eda-b520-c1a8d482cb1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:57.800841 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 935/1869] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:57.817956 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251a620-80a1-41bb-b705-fb06bdd74c9a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:57.819485 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5251a620-80a1-41bb-b705-fb06bdd74c9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:57.833223 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:57.878944 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251a620-80a1-41bb-b705-fb06bdd74c9a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:57.883094 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db496a13-522d-4577-a60f-34d6380c532f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:57.883721 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db496a13-522d-4577-a60f-34d6380c532f service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:57.884001 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db496a13-522d-4577-a60f-34d6380c532f service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:57.884273 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db496a13-522d-4577-a60f-34d6380c532f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:57.894300 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db496a13-522d-4577-a60f-34d6380c532f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:57.895801 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db496a13-522d-4577-a60f-34d6380c532f service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:57.905839 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 935/1870] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:26:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:26:58.194017 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8b0a1-b52c-4eda-b520-c1a8d482cb1a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:58.195533 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-05f8b0a1-b52c-4eda-b520-c1a8d482cb1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:58.210244 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8b0a1-b52c-4eda-b520-c1a8d482cb1a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:58.215238 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953aa09-b9b4-4c29-a8e5-a254e5326a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:58.215910 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953aa09-b9b4-4c29-a8e5-a254e5326a77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:58.216165 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953aa09-b9b4-4c29-a8e5-a254e5326a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:58.216440 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953aa09-b9b4-4c29-a8e5-a254e5326a77 admin admin] PATH_INFO: `/v3/projects/562d0d464a2b4385b8f4809b60789b7c` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:58.217636 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953aa09-b9b4-4c29-a8e5-a254e5326a77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=562d0d464a2b4385b8f4809b60789b7c)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:58.219597 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953aa09-b9b4-4c29-a8e5-a254e5326a77 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:58.272864 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8953aa09-b9b4-4c29-a8e5-a254e5326a77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '562d0d464a2b4385b8f4809b60789b7c', 'request_id': 'req-8953aa09-b9b4-4c29-a8e5-a254e5326a77'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:58.281688 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8953aa09-b9b4-4c29-a8e5-a254e5326a77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '562d0d464a2b4385b8f4809b60789b7c', 'request_id': 'req-8953aa09-b9b4-4c29-a8e5-a254e5326a77'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:58.288853 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8953aa09-b9b4-4c29-a8e5-a254e5326a77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '562d0d464a2b4385b8f4809b60789b7c', 'request_id': 'req-8953aa09-b9b4-4c29-a8e5-a254e5326a77'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:58.300399 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8953aa09-b9b4-4c29-a8e5-a254e5326a77 admin admin] The token cache is being invalidate because project 562d0d464a2b4385b8f4809b60789b7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:26:58.300688 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8953aa09-b9b4-4c29-a8e5-a254e5326a77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:58.301496 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8953aa09-b9b4-4c29-a8e5-a254e5326a77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:58.303099 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 936/1871] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:26:58 2022] DELETE /identity/v3/projects/562d0d464a2b4385b8f4809b60789b7c => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:26:58.307259 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db496a13-522d-4577-a60f-34d6380c532f service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:58.317181 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db496a13-522d-4577-a60f-34d6380c532f service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:58.376351 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b18cb4-450f-433e-b818-e9dd7767f9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:58.377035 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b18cb4-450f-433e-b818-e9dd7767f9c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:58.377319 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b18cb4-450f-433e-b818-e9dd7767f9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:58.377595 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b18cb4-450f-433e-b818-e9dd7767f9c0 admin admin] PATH_INFO: `/v3/users/4e67c38840a2436bb24b3cecdef1f2f6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:58.378773 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b18cb4-450f-433e-b818-e9dd7767f9c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e67c38840a2436bb24b3cecdef1f2f6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:58.380739 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b18cb4-450f-433e-b818-e9dd7767f9c0 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:58.434619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-40b18cb4-450f-433e-b818-e9dd7767f9c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e67c38840a2436bb24b3cecdef1f2f6'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:58.445480 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-40b18cb4-450f-433e-b818-e9dd7767f9c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e67c38840a2436bb24b3cecdef1f2f6'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:58.446258 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-40b18cb4-450f-433e-b818-e9dd7767f9c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e67c38840a2436bb24b3cecdef1f2f6'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:58.459410 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-40b18cb4-450f-433e-b818-e9dd7767f9c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e67c38840a2436bb24b3cecdef1f2f6'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:58.460133 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-40b18cb4-450f-433e-b818-e9dd7767f9c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e67c38840a2436bb24b3cecdef1f2f6'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:58.473392 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 936/1872] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:26:58 2022] DELETE /identity/v3/users/4e67c38840a2436bb24b3cecdef1f2f6 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:26:59.324636 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141d690-ff9c-4093-9358-061574950403 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:59.325138 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141d690-ff9c-4093-9358-061574950403 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:59.325213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141d690-ff9c-4093-9358-061574950403 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:59.326569 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 937/1873] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 5 10:26:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:26:59.346141 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67dfa3d-e6bb-4771-8d6e-8704195788ec None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:59.347082 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67dfa3d-e6bb-4771-8d6e-8704195788ec None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:59.348973 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67dfa3d-e6bb-4771-8d6e-8704195788ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:59.378338 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-f67dfa3d-e6bb-4771-8d6e-8704195788ec None None] MFA Rules not processed for user `b938f1f41db94b0eafe95bc8eeaae175`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:59.379822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f67dfa3d-e6bb-4771-8d6e-8704195788ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:59.428326 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:59.495280 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 937/1874] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 5 10:26:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:59.508534 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a141d690-ff9c-4093-9358-061574950403 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:59.510403 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a141d690-ff9c-4093-9358-061574950403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:59.531678 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a141d690-ff9c-4093-9358-061574950403 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:59.541023 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5185b5ac-ca66-438d-9f26-2c30df3d63d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:59.542082 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5185b5ac-ca66-438d-9f26-2c30df3d63d5 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:59.542528 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5185b5ac-ca66-438d-9f26-2c30df3d63d5 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:59.543149 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5185b5ac-ca66-438d-9f26-2c30df3d63d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:59.568490 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5185b5ac-ca66-438d-9f26-2c30df3d63d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:59.570358 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5185b5ac-ca66-438d-9f26-2c30df3d63d5 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:59.579313 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67dfa3d-e6bb-4771-8d6e-8704195788ec None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:59.580961 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f67dfa3d-e6bb-4771-8d6e-8704195788ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:59.582286 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 938/1875] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:26:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:26:59.671863 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67dfa3d-e6bb-4771-8d6e-8704195788ec None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:59.676828 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39285f97-0b0a-4e42-853c-144b858535eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:59.677494 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39285f97-0b0a-4e42-853c-144b858535eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:59.677774 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39285f97-0b0a-4e42-853c-144b858535eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:59.678042 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39285f97-0b0a-4e42-853c-144b858535eb admin admin] PATH_INFO: `/v3/projects/2e7e8cb23de249eb854a4c11eb8269f3` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:59.679248 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39285f97-0b0a-4e42-853c-144b858535eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e7e8cb23de249eb854a4c11eb8269f3)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:59.681181 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39285f97-0b0a-4e42-853c-144b858535eb admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:59.739301 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-39285f97-0b0a-4e42-853c-144b858535eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e7e8cb23de249eb854a4c11eb8269f3', 'request_id': 'req-39285f97-0b0a-4e42-853c-144b858535eb'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:59.745256 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-39285f97-0b0a-4e42-853c-144b858535eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e7e8cb23de249eb854a4c11eb8269f3', 'request_id': 'req-39285f97-0b0a-4e42-853c-144b858535eb'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:59.754261 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-39285f97-0b0a-4e42-853c-144b858535eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e7e8cb23de249eb854a4c11eb8269f3', 'request_id': 'req-39285f97-0b0a-4e42-853c-144b858535eb'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:59.769713 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-39285f97-0b0a-4e42-853c-144b858535eb admin admin] The token cache is being invalidate because project 2e7e8cb23de249eb854a4c11eb8269f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:26:59.769998 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-39285f97-0b0a-4e42-853c-144b858535eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:59.770727 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-39285f97-0b0a-4e42-853c-144b858535eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:26:59.772353 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 938/1876] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:26:59 2022] DELETE /identity/v3/projects/2e7e8cb23de249eb854a4c11eb8269f3 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:26:59.777448 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a4d958-b947-4b76-a710-948f2ce2d77e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:59.777709 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a4d958-b947-4b76-a710-948f2ce2d77e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:59.777987 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a4d958-b947-4b76-a710-948f2ce2d77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:59.799175 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d1a4d958-b947-4b76-a710-948f2ce2d77e None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:59.800364 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d1a4d958-b947-4b76-a710-948f2ce2d77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:59.861855 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 939/1877] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:59.870808 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bebb32e-bff9-4936-8950-51df7804543d None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:59.871148 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bebb32e-bff9-4936-8950-51df7804543d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:59.871427 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bebb32e-bff9-4936-8950-51df7804543d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:59.891693 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-8bebb32e-bff9-4936-8950-51df7804543d None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:26:59.893093 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8bebb32e-bff9-4936-8950-51df7804543d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:26:59.901294 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 939/1878] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:26:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:26:59.905708 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a4d958-b947-4b76-a710-948f2ce2d77e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:59.916187 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a4d958-b947-4b76-a710-948f2ce2d77e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:59.921176 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe485ff8-d2e2-457a-8951-fc5661d47020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:59.921711 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe485ff8-d2e2-457a-8951-fc5661d47020 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:59.921966 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe485ff8-d2e2-457a-8951-fc5661d47020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:59.922238 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe485ff8-d2e2-457a-8951-fc5661d47020 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:59.922996 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe485ff8-d2e2-457a-8951-fc5661d47020 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:26:59.923306 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe485ff8-d2e2-457a-8951-fc5661d47020 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:59.924905 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe485ff8-d2e2-457a-8951-fc5661d47020 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:59.934495 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:26:59.938255 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 940/1879] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:26:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:26:59.942991 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bebb32e-bff9-4936-8950-51df7804543d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:26:59.952303 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bebb32e-bff9-4936-8950-51df7804543d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:26:59.958006 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31078c6c-d990-4e12-9304-77b46b200e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:26:59.958727 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31078c6c-d990-4e12-9304-77b46b200e66 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:26:59.959080 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31078c6c-d990-4e12-9304-77b46b200e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:26:59.959440 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31078c6c-d990-4e12-9304-77b46b200e66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:26:59.960472 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31078c6c-d990-4e12-9304-77b46b200e66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:26:59.962789 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31078c6c-d990-4e12-9304-77b46b200e66 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:26:59.992869 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 940/1880] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:26:59 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:26:59.999525 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe485ff8-d2e2-457a-8951-fc5661d47020 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:00.010379 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe485ff8-d2e2-457a-8951-fc5661d47020 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:00.069256 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fd36cd-bbf9-41ab-9afd-13da63968a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:00.069870 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd36cd-bbf9-41ab-9afd-13da63968a04 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:00.070150 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd36cd-bbf9-41ab-9afd-13da63968a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:00.070422 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd36cd-bbf9-41ab-9afd-13da63968a04 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:00.071114 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fd36cd-bbf9-41ab-9afd-13da63968a04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:00.073036 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fd36cd-bbf9-41ab-9afd-13da63968a04 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:00.095999 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 941/1881] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:26:59 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:27:00.100240 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31078c6c-d990-4e12-9304-77b46b200e66 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:00.110597 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31078c6c-d990-4e12-9304-77b46b200e66 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:00.116365 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c108934-cc7b-414b-979e-60b5522a7fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:00.116981 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c108934-cc7b-414b-979e-60b5522a7fee admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:00.117257 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c108934-cc7b-414b-979e-60b5522a7fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:00.117489 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c108934-cc7b-414b-979e-60b5522a7fee admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:00.118203 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c108934-cc7b-414b-979e-60b5522a7fee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:00.119836 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c108934-cc7b-414b-979e-60b5522a7fee admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:00.130556 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 941/1882] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:27:00.135158 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fd36cd-bbf9-41ab-9afd-13da63968a04 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:00.152091 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fd36cd-bbf9-41ab-9afd-13da63968a04 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:00.158753 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383be7bf-42b9-4cb3-a26a-236097baeceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:00.159375 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383be7bf-42b9-4cb3-a26a-236097baeceb admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:00.159661 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383be7bf-42b9-4cb3-a26a-236097baeceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:00.159940 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383be7bf-42b9-4cb3-a26a-236097baeceb admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:00.161447 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383be7bf-42b9-4cb3-a26a-236097baeceb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:00.163042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383be7bf-42b9-4cb3-a26a-236097baeceb admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:00.175886 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 942/1883] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:27:00.180048 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c108934-cc7b-414b-979e-60b5522a7fee admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:00.190743 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c108934-cc7b-414b-979e-60b5522a7fee admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:00.195393 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12afa396-7287-4731-b9e3-ff578de19c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:00.196080 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12afa396-7287-4731-b9e3-ff578de19c58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:00.196364 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12afa396-7287-4731-b9e3-ff578de19c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:00.196672 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12afa396-7287-4731-b9e3-ff578de19c58 admin admin] PATH_INFO: `/v3/projects/3186acf85ed345deb6505112ff6f1909/users/33277dcb6bdf4e1b99afbc28ea2f68c3/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:00.209467 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:27:00.212684 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12afa396-7287-4731-b9e3-ff578de19c58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3186acf85ed345deb6505112ff6f1909, user_id=33277dcb6bdf4e1b99afbc28ea2f68c3, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:00.215990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12afa396-7287-4731-b9e3-ff578de19c58 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:00.239159 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 942/1884] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:27:00 2022] PUT /identity/v3/projects/3186acf85ed345deb6505112ff6f1909/users/33277dcb6bdf4e1b99afbc28ea2f68c3/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:27:00.241284 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75551c4e-059d-4c25-b8b7-9d9685bf9ee6 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:00.241641 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75551c4e-059d-4c25-b8b7-9d9685bf9ee6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:00.242721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75551c4e-059d-4c25-b8b7-9d9685bf9ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:00.264046 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-75551c4e-059d-4c25-b8b7-9d9685bf9ee6 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:27:00.265285 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-75551c4e-059d-4c25-b8b7-9d9685bf9ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:00.337941 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 943/1885] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:27:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:27:00.347921 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12afa396-7287-4731-b9e3-ff578de19c58 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:00.349772 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-12afa396-7287-4731-b9e3-ff578de19c58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:00.409095 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12afa396-7287-4731-b9e3-ff578de19c58 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:00.413210 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3913ba0-dd26-4106-b80a-5f71a005b49b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:00.413841 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3913ba0-dd26-4106-b80a-5f71a005b49b service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:00.414123 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3913ba0-dd26-4106-b80a-5f71a005b49b service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:00.414396 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3913ba0-dd26-4106-b80a-5f71a005b49b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:00.424711 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3913ba0-dd26-4106-b80a-5f71a005b49b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:00.426173 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3913ba0-dd26-4106-b80a-5f71a005b49b service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:00.436954 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 943/1886] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:27:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:27:01.170228 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ec3f2-3e00-421e-9009-1e332ec22801 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:01.170767 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ec3f2-3e00-421e-9009-1e332ec22801 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:01.172944 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ec3f2-3e00-421e-9009-1e332ec22801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:01.208383 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f11ec3f2-3e00-421e-9009-1e332ec22801 None None] MFA Rules not processed for user `33277dcb6bdf4e1b99afbc28ea2f68c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:27:01.209630 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f11ec3f2-3e00-421e-9009-1e332ec22801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:01.273956 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 944/1887] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:27:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:27:01.278658 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3913ba0-dd26-4106-b80a-5f71a005b49b service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:01.289195 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3913ba0-dd26-4106-b80a-5f71a005b49b service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:01.293972 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501ea9e8-0c84-468f-8cb5-ab14f52303ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:01.294591 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ea9e8-0c84-468f-8cb5-ab14f52303ab admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:01.294874 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ea9e8-0c84-468f-8cb5-ab14f52303ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:01.295237 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ea9e8-0c84-468f-8cb5-ab14f52303ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:01.295899 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501ea9e8-0c84-468f-8cb5-ab14f52303ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:01.297895 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501ea9e8-0c84-468f-8cb5-ab14f52303ab admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:01.327782 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 944/1888] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:27:01 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:27:01.332152 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11ec3f2-3e00-421e-9009-1e332ec22801 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:01.343238 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11ec3f2-3e00-421e-9009-1e332ec22801 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:01.350644 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:27:01.405474 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a2ff5-e144-4487-8633-887bf6def307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:01.406411 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a2ff5-e144-4487-8633-887bf6def307 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:01.406680 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a2ff5-e144-4487-8633-887bf6def307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:01.406972 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a2ff5-e144-4487-8633-887bf6def307 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:01.407812 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045a2ff5-e144-4487-8633-887bf6def307 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:01.409729 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045a2ff5-e144-4487-8633-887bf6def307 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:01.435709 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 945/1889] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:27:01 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:27:01.440097 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501ea9e8-0c84-468f-8cb5-ab14f52303ab admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:01.450856 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501ea9e8-0c84-468f-8cb5-ab14f52303ab admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:01.455903 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5ec48f-43d7-467e-9cd7-2b254b82a038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:01.456602 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ec48f-43d7-467e-9cd7-2b254b82a038 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:01.456946 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ec48f-43d7-467e-9cd7-2b254b82a038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:01.457282 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ec48f-43d7-467e-9cd7-2b254b82a038 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:01.458167 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5ec48f-43d7-467e-9cd7-2b254b82a038 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:01.459877 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5ec48f-43d7-467e-9cd7-2b254b82a038 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:01.474396 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 945/1890] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:27:01.478704 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a2ff5-e144-4487-8633-887bf6def307 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:01.489868 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a2ff5-e144-4487-8633-887bf6def307 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:01.495067 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04412fb6-2bf1-47f7-922f-8d096f3451f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:01.495752 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04412fb6-2bf1-47f7-922f-8d096f3451f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:01.496031 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04412fb6-2bf1-47f7-922f-8d096f3451f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:01.496312 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04412fb6-2bf1-47f7-922f-8d096f3451f2 admin admin] PATH_INFO: `/v3/projects/b636afcc425c432a9ca0ab3c41667ef7/users/3ec89ea8841d414aad41891f310538bd/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:01.512008 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04412fb6-2bf1-47f7-922f-8d096f3451f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b636afcc425c432a9ca0ab3c41667ef7, user_id=3ec89ea8841d414aad41891f310538bd, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:01.514667 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04412fb6-2bf1-47f7-922f-8d096f3451f2 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:01.529715 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 946/1891] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:27:01 2022] PUT /identity/v3/projects/b636afcc425c432a9ca0ab3c41667ef7/users/3ec89ea8841d414aad41891f310538bd/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:27:01.533568 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5ec48f-43d7-467e-9cd7-2b254b82a038 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:01.544103 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5ec48f-43d7-467e-9cd7-2b254b82a038 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:01.603599 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ecb11-5712-4262-9c10-597c9db970c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:01.604241 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ecb11-5712-4262-9c10-597c9db970c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:01.604692 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ecb11-5712-4262-9c10-597c9db970c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:01.604813 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ecb11-5712-4262-9c10-597c9db970c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:01.605489 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ecb11-5712-4262-9c10-597c9db970c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:01.608352 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ecb11-5712-4262-9c10-597c9db970c5 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:01.620639 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 946/1892] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 10:27:01.624828 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04412fb6-2bf1-47f7-922f-8d096f3451f2 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:01.651168 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04412fb6-2bf1-47f7-922f-8d096f3451f2 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:01.658886 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8269875-e861-4b53-9592-f780a981d70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:01.660166 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8269875-e861-4b53-9592-f780a981d70f admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:01.660873 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8269875-e861-4b53-9592-f780a981d70f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:01.661775 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8269875-e861-4b53-9592-f780a981d70f admin admin] PATH_INFO: `/v3/projects/b636afcc425c432a9ca0ab3c41667ef7/users/3ec89ea8841d414aad41891f310538bd/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:01.665322 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8269875-e861-4b53-9592-f780a981d70f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b636afcc425c432a9ca0ab3c41667ef7, user_id=3ec89ea8841d414aad41891f310538bd, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:01.668941 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8269875-e861-4b53-9592-f780a981d70f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:01.686863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 947/1893] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:27:01 2022] PUT /identity/v3/projects/b636afcc425c432a9ca0ab3c41667ef7/users/3ec89ea8841d414aad41891f310538bd/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:27:02.480061 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140da8b4-bd5e-4032-a117-927f804983ee None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:02.480588 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140da8b4-bd5e-4032-a117-927f804983ee None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:02.480677 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140da8b4-bd5e-4032-a117-927f804983ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:02.510856 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-140da8b4-bd5e-4032-a117-927f804983ee None None] MFA Rules not processed for user `3ec89ea8841d414aad41891f310538bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:27:02.512096 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-140da8b4-bd5e-4032-a117-927f804983ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:02.563717 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:27:02.592542 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 947/1894] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:27:02 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:27:02.604531 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8269875-e861-4b53-9592-f780a981d70f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:02.606552 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d8269875-e861-4b53-9592-f780a981d70f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:02.673759 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8269875-e861-4b53-9592-f780a981d70f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:02.677568 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d2a37-ea5c-48ee-b535-e0689f183a6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:02.678344 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d2a37-ea5c-48ee-b535-e0689f183a6a service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:02.678625 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d2a37-ea5c-48ee-b535-e0689f183a6a service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:02.678894 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d2a37-ea5c-48ee-b535-e0689f183a6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:02.737311 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4d2a37-ea5c-48ee-b535-e0689f183a6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:02.738882 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4d2a37-ea5c-48ee-b535-e0689f183a6a service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:02.749219 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 948/1895] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:27:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:27:05.188271 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140da8b4-bd5e-4032-a117-927f804983ee None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:05.190422 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-140da8b4-bd5e-4032-a117-927f804983ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:05.202581 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140da8b4-bd5e-4032-a117-927f804983ee None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:05.206845 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7749f4ac-9b7e-479e-b3b6-df6d8f76b571 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:05.207641 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749f4ac-9b7e-479e-b3b6-df6d8f76b571 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:05.207995 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749f4ac-9b7e-479e-b3b6-df6d8f76b571 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:05.208269 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749f4ac-9b7e-479e-b3b6-df6d8f76b571 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:05.218885 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7749f4ac-9b7e-479e-b3b6-df6d8f76b571 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:05.220443 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7749f4ac-9b7e-479e-b3b6-df6d8f76b571 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:05.231852 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 948/1896] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:27:05 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:27:07.203821 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d2a37-ea5c-48ee-b535-e0689f183a6a service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:07.206536 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5c4d2a37-ea5c-48ee-b535-e0689f183a6a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:07.228345 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:27:07.284151 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d2a37-ea5c-48ee-b535-e0689f183a6a service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:07.289906 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd9234-f3b2-474b-a61c-05fd9f7b867b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:07.290851 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd9234-f3b2-474b-a61c-05fd9f7b867b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:07.291250 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd9234-f3b2-474b-a61c-05fd9f7b867b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:07.291695 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd9234-f3b2-474b-a61c-05fd9f7b867b admin admin] PATH_INFO: `/v3/users/665c39df8a9649dd88eda36d2f16069e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:07.293087 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cd9234-f3b2-474b-a61c-05fd9f7b867b admin admin] RBAC: Authorizing `identity:delete_user(user_id=665c39df8a9649dd88eda36d2f16069e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:07.295200 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cd9234-f3b2-474b-a61c-05fd9f7b867b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:07.364923 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e8cd9234-f3b2-474b-a61c-05fd9f7b867b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '665c39df8a9649dd88eda36d2f16069e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:07.365560 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e8cd9234-f3b2-474b-a61c-05fd9f7b867b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '665c39df8a9649dd88eda36d2f16069e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:07.366463 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e8cd9234-f3b2-474b-a61c-05fd9f7b867b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '665c39df8a9649dd88eda36d2f16069e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:07.376975 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e8cd9234-f3b2-474b-a61c-05fd9f7b867b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '665c39df8a9649dd88eda36d2f16069e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:07.386480 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e8cd9234-f3b2-474b-a61c-05fd9f7b867b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '665c39df8a9649dd88eda36d2f16069e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:07.408297 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 949/1897] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:27:07 2022] DELETE /identity/v3/users/665c39df8a9649dd88eda36d2f16069e => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:27:07.413652 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29240031-7b4c-404b-b06f-66e20e30679a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:07.414185 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29240031-7b4c-404b-b06f-66e20e30679a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:07.414836 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29240031-7b4c-404b-b06f-66e20e30679a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:07.441633 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-29240031-7b4c-404b-b06f-66e20e30679a None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:27:07.443005 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-29240031-7b4c-404b-b06f-66e20e30679a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:07.526524 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 949/1898] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:27:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:27:07.537388 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd9234-f3b2-474b-a61c-05fd9f7b867b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:07.538800 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e8cd9234-f3b2-474b-a61c-05fd9f7b867b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:07.600782 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd9234-f3b2-474b-a61c-05fd9f7b867b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:07.604845 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513eb8b8-26ba-4235-b032-e2909fbafc0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:07.605483 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513eb8b8-26ba-4235-b032-e2909fbafc0f service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:07.605763 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513eb8b8-26ba-4235-b032-e2909fbafc0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:07.606034 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513eb8b8-26ba-4235-b032-e2909fbafc0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:07.615769 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513eb8b8-26ba-4235-b032-e2909fbafc0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:07.617212 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513eb8b8-26ba-4235-b032-e2909fbafc0f service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:07.627107 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 950/1899] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:27:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:27:08.385960 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29240031-7b4c-404b-b06f-66e20e30679a None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:08.388294 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-29240031-7b4c-404b-b06f-66e20e30679a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:08.439663 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29240031-7b4c-404b-b06f-66e20e30679a None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:08.454630 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa883df2-d29f-4c65-8602-63b48c46230a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:08.455432 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa883df2-d29f-4c65-8602-63b48c46230a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:08.456611 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa883df2-d29f-4c65-8602-63b48c46230a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:08.457052 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa883df2-d29f-4c65-8602-63b48c46230a admin admin] PATH_INFO: `/v3/projects/87d0ad2289bc4df88b568265e3ed8905` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:08.459190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa883df2-d29f-4c65-8602-63b48c46230a admin admin] RBAC: Authorizing `identity:delete_project(project_id=87d0ad2289bc4df88b568265e3ed8905)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:08.463904 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa883df2-d29f-4c65-8602-63b48c46230a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:08.574923 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-fa883df2-d29f-4c65-8602-63b48c46230a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87d0ad2289bc4df88b568265e3ed8905', 'request_id': 'req-fa883df2-d29f-4c65-8602-63b48c46230a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:08.583395 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-fa883df2-d29f-4c65-8602-63b48c46230a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87d0ad2289bc4df88b568265e3ed8905', 'request_id': 'req-fa883df2-d29f-4c65-8602-63b48c46230a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:08.598608 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-fa883df2-d29f-4c65-8602-63b48c46230a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87d0ad2289bc4df88b568265e3ed8905', 'request_id': 'req-fa883df2-d29f-4c65-8602-63b48c46230a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:08.616791 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-fa883df2-d29f-4c65-8602-63b48c46230a admin admin] The token cache is being invalidate because project 87d0ad2289bc4df88b568265e3ed8905 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:27:08.617320 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-fa883df2-d29f-4c65-8602-63b48c46230a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:08.618310 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-fa883df2-d29f-4c65-8602-63b48c46230a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:08.620279 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 950/1900] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:27:08 2022] DELETE /identity/v3/projects/87d0ad2289bc4df88b568265e3ed8905 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:27:08.624415 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513eb8b8-26ba-4235-b032-e2909fbafc0f service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:08.635441 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513eb8b8-26ba-4235-b032-e2909fbafc0f service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:08.685342 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:27:08.721005 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfbbea6-7fcd-425a-869a-e962c189c543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:08.721736 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfbbea6-7fcd-425a-869a-e962c189c543 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:08.722024 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfbbea6-7fcd-425a-869a-e962c189c543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:08.722299 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfbbea6-7fcd-425a-869a-e962c189c543 admin admin] PATH_INFO: `/v3/users/4ef3eeb784cd4d81a61d0def9de98c3b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:08.723499 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfbbea6-7fcd-425a-869a-e962c189c543 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ef3eeb784cd4d81a61d0def9de98c3b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:08.725460 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfbbea6-7fcd-425a-869a-e962c189c543 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:08.775247 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5bfbbea6-7fcd-425a-869a-e962c189c543 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ef3eeb784cd4d81a61d0def9de98c3b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:08.777034 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5bfbbea6-7fcd-425a-869a-e962c189c543 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ef3eeb784cd4d81a61d0def9de98c3b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:08.777726 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5bfbbea6-7fcd-425a-869a-e962c189c543 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ef3eeb784cd4d81a61d0def9de98c3b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:08.785902 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5bfbbea6-7fcd-425a-869a-e962c189c543 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ef3eeb784cd4d81a61d0def9de98c3b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:08.794367 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5bfbbea6-7fcd-425a-869a-e962c189c543 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ef3eeb784cd4d81a61d0def9de98c3b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:08.811953 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 951/1901] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:27:08 2022] DELETE /identity/v3/users/4ef3eeb784cd4d81a61d0def9de98c3b => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:27:09.138005 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa883df2-d29f-4c65-8602-63b48c46230a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:09.139507 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fa883df2-d29f-4c65-8602-63b48c46230a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:09.163080 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa883df2-d29f-4c65-8602-63b48c46230a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:09.214784 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:27:09.224942 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f370b-78e3-47ff-8857-f1db8e4ce4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:09.225737 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f370b-78e3-47ff-8857-f1db8e4ce4a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:09.226402 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f370b-78e3-47ff-8857-f1db8e4ce4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:09.226755 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f370b-78e3-47ff-8857-f1db8e4ce4a2 admin admin] PATH_INFO: `/v3/projects/5e0f3a5fc6eb42a5b93c3a745e14d6d1` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:09.228022 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f370b-78e3-47ff-8857-f1db8e4ce4a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e0f3a5fc6eb42a5b93c3a745e14d6d1)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:09.230299 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f370b-78e3-47ff-8857-f1db8e4ce4a2 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:09.286413 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3e9f370b-78e3-47ff-8857-f1db8e4ce4a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e0f3a5fc6eb42a5b93c3a745e14d6d1', 'request_id': 'req-3e9f370b-78e3-47ff-8857-f1db8e4ce4a2'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:09.291931 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3e9f370b-78e3-47ff-8857-f1db8e4ce4a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e0f3a5fc6eb42a5b93c3a745e14d6d1', 'request_id': 'req-3e9f370b-78e3-47ff-8857-f1db8e4ce4a2'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:09.301825 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3e9f370b-78e3-47ff-8857-f1db8e4ce4a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e0f3a5fc6eb42a5b93c3a745e14d6d1', 'request_id': 'req-3e9f370b-78e3-47ff-8857-f1db8e4ce4a2'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:09.314318 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3e9f370b-78e3-47ff-8857-f1db8e4ce4a2 admin admin] The token cache is being invalidate because project 5e0f3a5fc6eb42a5b93c3a745e14d6d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:27:09.314634 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3e9f370b-78e3-47ff-8857-f1db8e4ce4a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:09.315564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3e9f370b-78e3-47ff-8857-f1db8e4ce4a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:09.318426 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 951/1902] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:27:09 2022] DELETE /identity/v3/projects/5e0f3a5fc6eb42a5b93c3a745e14d6d1 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:27:09.323488 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a85b9-13b6-481d-a250-37b78c870fb3 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:09.323883 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a85b9-13b6-481d-a250-37b78c870fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:09.324301 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a85b9-13b6-481d-a250-37b78c870fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:09.345786 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-004a85b9-13b6-481d-a250-37b78c870fb3 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:27:09.347092 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-004a85b9-13b6-481d-a250-37b78c870fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:09.447934 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 952/1903] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:27:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:27:09.456747 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2445e78-e613-41e3-a821-241bef9c0e59 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:09.457280 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2445e78-e613-41e3-a821-241bef9c0e59 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:09.457280 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2445e78-e613-41e3-a821-241bef9c0e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:09.505213 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-f2445e78-e613-41e3-a821-241bef9c0e59 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:27:09.506785 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f2445e78-e613-41e3-a821-241bef9c0e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:09.517986 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 952/1904] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:27:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:27:09.523874 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004a85b9-13b6-481d-a250-37b78c870fb3 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:09.535040 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004a85b9-13b6-481d-a250-37b78c870fb3 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:09.540510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca42f25d-9507-41c4-b8fc-9e4cbe400bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:09.541203 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca42f25d-9507-41c4-b8fc-9e4cbe400bc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:09.541505 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca42f25d-9507-41c4-b8fc-9e4cbe400bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:09.541780 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca42f25d-9507-41c4-b8fc-9e4cbe400bc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:09.542797 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca42f25d-9507-41c4-b8fc-9e4cbe400bc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:27:09.543172 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca42f25d-9507-41c4-b8fc-9e4cbe400bc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:09.544982 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca42f25d-9507-41c4-b8fc-9e4cbe400bc3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:09.563206 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 953/1905] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:27:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:27:09.567429 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2445e78-e613-41e3-a821-241bef9c0e59 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:09.578457 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2445e78-e613-41e3-a821-241bef9c0e59 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:09.583745 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca0c2a2-507e-4bdf-8baf-b069a46b7b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:09.584377 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0c2a2-507e-4bdf-8baf-b069a46b7b21 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:09.584778 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0c2a2-507e-4bdf-8baf-b069a46b7b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:09.585238 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0c2a2-507e-4bdf-8baf-b069a46b7b21 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:09.585907 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca0c2a2-507e-4bdf-8baf-b069a46b7b21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:09.587872 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca0c2a2-507e-4bdf-8baf-b069a46b7b21 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:09.617619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 953/1906] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:27:09 2022] POST /identity/v3/projects => generated 399 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:27:09.621693 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca42f25d-9507-41c4-b8fc-9e4cbe400bc3 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:09.633642 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca42f25d-9507-41c4-b8fc-9e4cbe400bc3 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:09.681338 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:27:09.708910 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621038a-42cf-4453-b9ac-209173caeeb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:09.709936 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621038a-42cf-4453-b9ac-209173caeeb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:09.710219 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621038a-42cf-4453-b9ac-209173caeeb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:09.710492 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621038a-42cf-4453-b9ac-209173caeeb2 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:09.712169 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d621038a-42cf-4453-b9ac-209173caeeb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:09.714312 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d621038a-42cf-4453-b9ac-209173caeeb2 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:09.756649 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 954/1907] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:27:09 2022] POST /identity/v3/users => generated 300 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:27:09.758838 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca0c2a2-507e-4bdf-8baf-b069a46b7b21 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:09.769772 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca0c2a2-507e-4bdf-8baf-b069a46b7b21 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:09.780622 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6239a5-00be-47ba-98ee-7ccafdc6d4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:09.781530 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6239a5-00be-47ba-98ee-7ccafdc6d4bd admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:09.781807 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6239a5-00be-47ba-98ee-7ccafdc6d4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:09.782358 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6239a5-00be-47ba-98ee-7ccafdc6d4bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:09.783253 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6239a5-00be-47ba-98ee-7ccafdc6d4bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:09.788893 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6239a5-00be-47ba-98ee-7ccafdc6d4bd admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:09.800873 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 954/1908] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 10:27:09.804983 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621038a-42cf-4453-b9ac-209173caeeb2 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:09.817104 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621038a-42cf-4453-b9ac-209173caeeb2 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:09.822228 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf7420-7a28-4e28-a7fe-6cf7e498e616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:09.822993 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf7420-7a28-4e28-a7fe-6cf7e498e616 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:09.823621 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf7420-7a28-4e28-a7fe-6cf7e498e616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:09.823978 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf7420-7a28-4e28-a7fe-6cf7e498e616 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:09.824778 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cf7420-7a28-4e28-a7fe-6cf7e498e616 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:09.826492 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cf7420-7a28-4e28-a7fe-6cf7e498e616 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:09.852464 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6239a5-00be-47ba-98ee-7ccafdc6d4bd admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:09.855950 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 955/1909] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:27:09.874405 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6239a5-00be-47ba-98ee-7ccafdc6d4bd admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:09.881870 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e6f8c-dd91-4155-af4b-1f8fd6c312f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:09.885195 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e6f8c-dd91-4155-af4b-1f8fd6c312f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:09.885829 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e6f8c-dd91-4155-af4b-1f8fd6c312f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:09.886284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e6f8c-dd91-4155-af4b-1f8fd6c312f5 admin admin] PATH_INFO: `/v3/projects/ff23649d09b6425083d1e306fd45afa2/users/a9a0d54db95949f3a6ccd78052c42371/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:09.912987 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619e6f8c-dd91-4155-af4b-1f8fd6c312f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff23649d09b6425083d1e306fd45afa2, user_id=a9a0d54db95949f3a6ccd78052c42371, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:09.916215 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619e6f8c-dd91-4155-af4b-1f8fd6c312f5 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:09.936743 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 955/1910] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:27:09 2022] PUT /identity/v3/projects/ff23649d09b6425083d1e306fd45afa2/users/a9a0d54db95949f3a6ccd78052c42371/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:27:09.949280 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa57c67-886c-4b0a-afe9-d4a0569694f0 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:09.949716 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa57c67-886c-4b0a-afe9-d4a0569694f0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:09.950103 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa57c67-886c-4b0a-afe9-d4a0569694f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:10.006258 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-3aa57c67-886c-4b0a-afe9-d4a0569694f0 None None] MFA Rules not processed for user `a9a0d54db95949f3a6ccd78052c42371`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:27:10.008007 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3aa57c67-886c-4b0a-afe9-d4a0569694f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:10.086518 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 956/1911] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:27:09 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:27:10.099033 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e6f8c-dd91-4155-af4b-1f8fd6c312f5 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:10.100713 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-619e6f8c-dd91-4155-af4b-1f8fd6c312f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:10.192169 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e6f8c-dd91-4155-af4b-1f8fd6c312f5 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:10.198736 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf496bc-bcf4-4edc-905c-1347dc8765a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:10.199631 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf496bc-bcf4-4edc-905c-1347dc8765a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:10.199991 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf496bc-bcf4-4edc-905c-1347dc8765a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:10.200319 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf496bc-bcf4-4edc-905c-1347dc8765a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:10.212254 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf496bc-bcf4-4edc-905c-1347dc8765a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:10.213754 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf496bc-bcf4-4edc-905c-1347dc8765a7 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:10.231056 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 956/1912] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:27:10 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:27:20.083163 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=46,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:27:20.220315 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=80,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:27:23.632458 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa57c67-886c-4b0a-afe9-d4a0569694f0 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:23.721739 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa57c67-886c-4b0a-afe9-d4a0569694f0 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:23.726454 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66e0bac-749c-4a41-81bc-ba30abf42b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:23.727331 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e0bac-749c-4a41-81bc-ba30abf42b56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:23.728016 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e0bac-749c-4a41-81bc-ba30abf42b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:23.728330 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e0bac-749c-4a41-81bc-ba30abf42b56 admin admin] PATH_INFO: `/v3/users/a9a0d54db95949f3a6ccd78052c42371` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:23.729539 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66e0bac-749c-4a41-81bc-ba30abf42b56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9a0d54db95949f3a6ccd78052c42371)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:23.731518 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66e0bac-749c-4a41-81bc-ba30abf42b56 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:23.802427 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a66e0bac-749c-4a41-81bc-ba30abf42b56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9a0d54db95949f3a6ccd78052c42371'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:23.803702 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a66e0bac-749c-4a41-81bc-ba30abf42b56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9a0d54db95949f3a6ccd78052c42371'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:23.805910 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a66e0bac-749c-4a41-81bc-ba30abf42b56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9a0d54db95949f3a6ccd78052c42371'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:23.820023 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a66e0bac-749c-4a41-81bc-ba30abf42b56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9a0d54db95949f3a6ccd78052c42371'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:23.834963 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a66e0bac-749c-4a41-81bc-ba30abf42b56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9a0d54db95949f3a6ccd78052c42371'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:23.893306 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 957/1913] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:27:23 2022] DELETE /identity/v3/users/a9a0d54db95949f3a6ccd78052c42371 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:27:23.895143 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b00332-1891-44c9-a5ba-6bbb37bf918a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:23.913758 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b00332-1891-44c9-a5ba-6bbb37bf918a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:23.914113 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b00332-1891-44c9-a5ba-6bbb37bf918a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:23.956132 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-98b00332-1891-44c9-a5ba-6bbb37bf918a None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:27:23.957978 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-98b00332-1891-44c9-a5ba-6bbb37bf918a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:24.107392 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 957/1914] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:27:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:27:24.134872 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66e0bac-749c-4a41-81bc-ba30abf42b56 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:24.134872 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a66e0bac-749c-4a41-81bc-ba30abf42b56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:24.360054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66e0bac-749c-4a41-81bc-ba30abf42b56 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:24.371732 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37bf6af-9ebd-486d-bf8f-dc7b8e75a389 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:24.372793 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37bf6af-9ebd-486d-bf8f-dc7b8e75a389 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:24.374699 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37bf6af-9ebd-486d-bf8f-dc7b8e75a389 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:24.374699 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37bf6af-9ebd-486d-bf8f-dc7b8e75a389 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:24.389118 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37bf6af-9ebd-486d-bf8f-dc7b8e75a389 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:24.392125 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37bf6af-9ebd-486d-bf8f-dc7b8e75a389 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:24.422710 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 958/1915] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:27:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:27:24.811840 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b00332-1891-44c9-a5ba-6bbb37bf918a None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:24.813306 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-98b00332-1891-44c9-a5ba-6bbb37bf918a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:24.835316 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b00332-1891-44c9-a5ba-6bbb37bf918a None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:24.843629 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8497e204-50da-4939-9be8-80bf63fc2df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:24.844332 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497e204-50da-4939-9be8-80bf63fc2df4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:24.844617 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497e204-50da-4939-9be8-80bf63fc2df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:24.844888 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497e204-50da-4939-9be8-80bf63fc2df4 admin admin] PATH_INFO: `/v3/projects/ff23649d09b6425083d1e306fd45afa2` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:24.846559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8497e204-50da-4939-9be8-80bf63fc2df4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff23649d09b6425083d1e306fd45afa2)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:24.848520 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8497e204-50da-4939-9be8-80bf63fc2df4 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:24.912636 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8497e204-50da-4939-9be8-80bf63fc2df4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff23649d09b6425083d1e306fd45afa2', 'request_id': 'req-8497e204-50da-4939-9be8-80bf63fc2df4'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:24.927390 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8497e204-50da-4939-9be8-80bf63fc2df4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff23649d09b6425083d1e306fd45afa2', 'request_id': 'req-8497e204-50da-4939-9be8-80bf63fc2df4'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:24.936645 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8497e204-50da-4939-9be8-80bf63fc2df4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff23649d09b6425083d1e306fd45afa2', 'request_id': 'req-8497e204-50da-4939-9be8-80bf63fc2df4'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:24.948422 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8497e204-50da-4939-9be8-80bf63fc2df4 admin admin] The token cache is being invalidate because project ff23649d09b6425083d1e306fd45afa2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:27:24.948711 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8497e204-50da-4939-9be8-80bf63fc2df4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:24.949378 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8497e204-50da-4939-9be8-80bf63fc2df4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:24.950950 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 958/1916] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:27:24 2022] DELETE /identity/v3/projects/ff23649d09b6425083d1e306fd45afa2 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:27:24.956160 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c4132-b63d-4f69-b7e9-542fcbd41472 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:24.956695 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c4132-b63d-4f69-b7e9-542fcbd41472 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:24.956695 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c4132-b63d-4f69-b7e9-542fcbd41472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:24.977761 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-554c4132-b63d-4f69-b7e9-542fcbd41472 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:27:24.979505 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-554c4132-b63d-4f69-b7e9-542fcbd41472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:24.999307 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:27:25.046778 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 959/1917] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:27:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:27:25.054881 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8fc1bb-cffe-49fc-8090-be0c3c172c49 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:25.056345 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8fc1bb-cffe-49fc-8090-be0c3c172c49 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:25.056638 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8fc1bb-cffe-49fc-8090-be0c3c172c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:25.082976 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-fe8fc1bb-cffe-49fc-8090-be0c3c172c49 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:27:25.084142 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fe8fc1bb-cffe-49fc-8090-be0c3c172c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:25.097489 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 959/1918] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:27:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:27:25.101282 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554c4132-b63d-4f69-b7e9-542fcbd41472 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:25.112564 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554c4132-b63d-4f69-b7e9-542fcbd41472 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:25.115269 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8fc1bb-cffe-49fc-8090-be0c3c172c49 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:25.116811 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fe8fc1bb-cffe-49fc-8090-be0c3c172c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:25.117935 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b458a9-3066-4e8b-b292-23fc5f456bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:25.118624 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b458a9-3066-4e8b-b292-23fc5f456bef admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:25.118922 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b458a9-3066-4e8b-b292-23fc5f456bef admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:25.119200 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b458a9-3066-4e8b-b292-23fc5f456bef admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:25.119906 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b458a9-3066-4e8b-b292-23fc5f456bef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:27:25.120200 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b458a9-3066-4e8b-b292-23fc5f456bef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:25.121887 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b458a9-3066-4e8b-b292-23fc5f456bef admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:25.129019 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8fc1bb-cffe-49fc-8090-be0c3c172c49 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:25.134156 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6f4c95-4180-4070-b522-19cc2067f7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:25.134813 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6f4c95-4180-4070-b522-19cc2067f7aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:25.135289 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6f4c95-4180-4070-b522-19cc2067f7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:25.135507 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6f4c95-4180-4070-b522-19cc2067f7aa admin admin] PATH_INFO: `/v3/users/869e4ed766e74503a54fe3197fb641b6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:25.137882 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 960/1919] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:27:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:27:25.143005 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b458a9-3066-4e8b-b292-23fc5f456bef admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:25.147425 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6f4c95-4180-4070-b522-19cc2067f7aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=869e4ed766e74503a54fe3197fb641b6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:25.149633 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6f4c95-4180-4070-b522-19cc2067f7aa admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:25.154633 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b458a9-3066-4e8b-b292-23fc5f456bef admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:25.164703 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e9b8e3-daf8-4ba6-ad80-5619dd588ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:25.165496 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9b8e3-daf8-4ba6-ad80-5619dd588ef3 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:25.165878 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9b8e3-daf8-4ba6-ad80-5619dd588ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:25.166320 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9b8e3-daf8-4ba6-ad80-5619dd588ef3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:25.167248 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e9b8e3-daf8-4ba6-ad80-5619dd588ef3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:25.169501 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e9b8e3-daf8-4ba6-ad80-5619dd588ef3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:25.202543 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 961/1920] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:27:25 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:27:25.207708 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e9b8e3-daf8-4ba6-ad80-5619dd588ef3 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:25.227520 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-db6f4c95-4180-4070-b522-19cc2067f7aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '869e4ed766e74503a54fe3197fb641b6'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:25.239044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-db6f4c95-4180-4070-b522-19cc2067f7aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '869e4ed766e74503a54fe3197fb641b6'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:25.240495 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-db6f4c95-4180-4070-b522-19cc2067f7aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '869e4ed766e74503a54fe3197fb641b6'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:25.260490 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-db6f4c95-4180-4070-b522-19cc2067f7aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '869e4ed766e74503a54fe3197fb641b6'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:25.262945 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-db6f4c95-4180-4070-b522-19cc2067f7aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '869e4ed766e74503a54fe3197fb641b6'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:25.266351 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:27:25.289600 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 960/1921] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:27:25 2022] DELETE /identity/v3/users/869e4ed766e74503a54fe3197fb641b6 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:27:25.295159 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2943256d-a6cc-4e55-b854-ee3e47ee80a0 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:25.295515 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2943256d-a6cc-4e55-b854-ee3e47ee80a0 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:25.295854 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2943256d-a6cc-4e55-b854-ee3e47ee80a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:25.312205 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e9b8e3-daf8-4ba6-ad80-5619dd588ef3 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:25.315801 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-2943256d-a6cc-4e55-b854-ee3e47ee80a0 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:27:25.317143 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2943256d-a6cc-4e55-b854-ee3e47ee80a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:25.378220 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88205d04-51e3-4f79-9001-5898dac783c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:25.378999 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88205d04-51e3-4f79-9001-5898dac783c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:25.379346 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88205d04-51e3-4f79-9001-5898dac783c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:25.379679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88205d04-51e3-4f79-9001-5898dac783c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:25.380419 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88205d04-51e3-4f79-9001-5898dac783c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:25.383288 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88205d04-51e3-4f79-9001-5898dac783c8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:25.421971 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 962/1922] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:27:25 2022] POST /identity/v3/users => generated 292 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:27:25.423576 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88205d04-51e3-4f79-9001-5898dac783c8 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:25.428861 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-88205d04-51e3-4f79-9001-5898dac783c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:25.429842 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 961/1923] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:27:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:27:25.447460 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2943256d-a6cc-4e55-b854-ee3e47ee80a0 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:25.447460 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2943256d-a6cc-4e55-b854-ee3e47ee80a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:25.455099 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88205d04-51e3-4f79-9001-5898dac783c8 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:25.462284 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e17a24-e6eb-4648-a9eb-22fdd0ceb6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:25.463305 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e17a24-e6eb-4648-a9eb-22fdd0ceb6e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:25.463772 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e17a24-e6eb-4648-a9eb-22fdd0ceb6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:25.464235 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e17a24-e6eb-4648-a9eb-22fdd0ceb6e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:25.465463 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e17a24-e6eb-4648-a9eb-22fdd0ceb6e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:25.468420 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e17a24-e6eb-4648-a9eb-22fdd0ceb6e3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:25.481214 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 963/1924] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:27:25.486533 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e17a24-e6eb-4648-a9eb-22fdd0ceb6e3 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:25.511445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e17a24-e6eb-4648-a9eb-22fdd0ceb6e3 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:25.522543 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ce8fbd-3c73-459a-b0e2-1264db750afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:25.523465 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce8fbd-3c73-459a-b0e2-1264db750afb admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:25.523757 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce8fbd-3c73-459a-b0e2-1264db750afb admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:25.524024 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce8fbd-3c73-459a-b0e2-1264db750afb admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:25.524821 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ce8fbd-3c73-459a-b0e2-1264db750afb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:25.527075 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ce8fbd-3c73-459a-b0e2-1264db750afb admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:25.534002 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2943256d-a6cc-4e55-b854-ee3e47ee80a0 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:25.539366 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e20b1-5dfa-4d78-9696-d9fa27427640 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:25.540000 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e20b1-5dfa-4d78-9696-d9fa27427640 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:25.540410 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e20b1-5dfa-4d78-9696-d9fa27427640 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:25.540815 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e20b1-5dfa-4d78-9696-d9fa27427640 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:25.542470 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 964/1925] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 10:27:25.546451 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ce8fbd-3c73-459a-b0e2-1264db750afb admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:25.553291 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:27:25.557674 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e20b1-5dfa-4d78-9696-d9fa27427640 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:25.559311 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ce8fbd-3c73-459a-b0e2-1264db750afb admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:25.559436 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e20b1-5dfa-4d78-9696-d9fa27427640 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:25.565582 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67df7869-310d-43a4-8fb4-83b4e82a540f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:25.566494 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df7869-310d-43a4-8fb4-83b4e82a540f admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:25.567202 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df7869-310d-43a4-8fb4-83b4e82a540f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:25.567657 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df7869-310d-43a4-8fb4-83b4e82a540f admin admin] PATH_INFO: `/v3/projects/db2897585ac048ae93df3e38d1437e6e/users/9697af06b1104425aed6692ba4f29238/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:25.571211 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 962/1926] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:27:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 10:27:25.590987 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67df7869-310d-43a4-8fb4-83b4e82a540f admin admin] RBAC: Authorizing `identity:create_grant(project_id=db2897585ac048ae93df3e38d1437e6e, user_id=9697af06b1104425aed6692ba4f29238, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:25.597746 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67df7869-310d-43a4-8fb4-83b4e82a540f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:25.624947 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 965/1927] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:27:25 2022] PUT /identity/v3/projects/db2897585ac048ae93df3e38d1437e6e/users/9697af06b1104425aed6692ba4f29238/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:27:25.626743 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae24bb-6092-440e-ac65-f27502c11314 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:25.627129 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae24bb-6092-440e-ac65-f27502c11314 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:25.627477 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae24bb-6092-440e-ac65-f27502c11314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:25.664614 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-02ae24bb-6092-440e-ac65-f27502c11314 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:27:25.668059 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-02ae24bb-6092-440e-ac65-f27502c11314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:25.739788 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 963/1928] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:27:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:27:25.761287 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67df7869-310d-43a4-8fb4-83b4e82a540f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:25.834794 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67df7869-310d-43a4-8fb4-83b4e82a540f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:25.839208 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf7e870-f56c-47e2-8490-9e17a3e8039f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:25.839844 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7e870-f56c-47e2-8490-9e17a3e8039f service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:25.840177 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7e870-f56c-47e2-8490-9e17a3e8039f service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:25.840403 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7e870-f56c-47e2-8490-9e17a3e8039f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:25.850072 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf7e870-f56c-47e2-8490-9e17a3e8039f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:25.851560 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf7e870-f56c-47e2-8490-9e17a3e8039f service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:25.862319 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 966/1929] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:27:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:27:25.977234 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ae24bb-6092-440e-ac65-f27502c11314 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:25.979045 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-02ae24bb-6092-440e-ac65-f27502c11314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:25.992613 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ae24bb-6092-440e-ac65-f27502c11314 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:25.997456 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d307392-5615-46a3-a6bc-bced8cf9b2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:25.998418 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d307392-5615-46a3-a6bc-bced8cf9b2b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:25.998720 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d307392-5615-46a3-a6bc-bced8cf9b2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:25.999095 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d307392-5615-46a3-a6bc-bced8cf9b2b1 admin admin] PATH_INFO: `/v3/projects/4f960ffa0fa3408fac7efa063d56054d` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:26.000383 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d307392-5615-46a3-a6bc-bced8cf9b2b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f960ffa0fa3408fac7efa063d56054d)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:26.002471 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d307392-5615-46a3-a6bc-bced8cf9b2b1 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:26.046152 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:27:26.082252 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0d307392-5615-46a3-a6bc-bced8cf9b2b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f960ffa0fa3408fac7efa063d56054d', 'request_id': 'req-0d307392-5615-46a3-a6bc-bced8cf9b2b1'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:26.094778 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0d307392-5615-46a3-a6bc-bced8cf9b2b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f960ffa0fa3408fac7efa063d56054d', 'request_id': 'req-0d307392-5615-46a3-a6bc-bced8cf9b2b1'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:26.109168 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0d307392-5615-46a3-a6bc-bced8cf9b2b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f960ffa0fa3408fac7efa063d56054d', 'request_id': 'req-0d307392-5615-46a3-a6bc-bced8cf9b2b1'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:26.125288 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0d307392-5615-46a3-a6bc-bced8cf9b2b1 admin admin] The token cache is being invalidate because project 4f960ffa0fa3408fac7efa063d56054d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:27:26.125581 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0d307392-5615-46a3-a6bc-bced8cf9b2b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:26.126398 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0d307392-5615-46a3-a6bc-bced8cf9b2b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:26.128585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 964/1930] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:27:25 2022] DELETE /identity/v3/projects/4f960ffa0fa3408fac7efa063d56054d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:27:26.132905 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf7e870-f56c-47e2-8490-9e17a3e8039f service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:26.143964 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf7e870-f56c-47e2-8490-9e17a3e8039f service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:26.238456 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08daba2f-18ee-4712-aa36-c64a08235f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:26.239903 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08daba2f-18ee-4712-aa36-c64a08235f91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:26.239903 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08daba2f-18ee-4712-aa36-c64a08235f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:26.239903 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08daba2f-18ee-4712-aa36-c64a08235f91 admin admin] PATH_INFO: `/v3/users/e6f9b803b9ef4b4b97abf62ab0369514` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:26.241388 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08daba2f-18ee-4712-aa36-c64a08235f91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6f9b803b9ef4b4b97abf62ab0369514)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:26.244634 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08daba2f-18ee-4712-aa36-c64a08235f91 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:26.304938 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-08daba2f-18ee-4712-aa36-c64a08235f91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6f9b803b9ef4b4b97abf62ab0369514'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:26.305624 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-08daba2f-18ee-4712-aa36-c64a08235f91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6f9b803b9ef4b4b97abf62ab0369514'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:26.306305 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-08daba2f-18ee-4712-aa36-c64a08235f91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6f9b803b9ef4b4b97abf62ab0369514'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:26.318446 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-08daba2f-18ee-4712-aa36-c64a08235f91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6f9b803b9ef4b4b97abf62ab0369514'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:26.328500 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-08daba2f-18ee-4712-aa36-c64a08235f91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6f9b803b9ef4b4b97abf62ab0369514'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:26.345047 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:27:26.350114 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 967/1931] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:27:26 2022] DELETE /identity/v3/users/e6f9b803b9ef4b4b97abf62ab0369514 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:27:26.769544 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d307392-5615-46a3-a6bc-bced8cf9b2b1 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:26.771332 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0d307392-5615-46a3-a6bc-bced8cf9b2b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:26.804799 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d307392-5615-46a3-a6bc-bced8cf9b2b1 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:26.893390 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3932e4e8-c1fb-4bef-b74d-6317bae1f805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:26.894055 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932e4e8-c1fb-4bef-b74d-6317bae1f805 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:26.894338 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932e4e8-c1fb-4bef-b74d-6317bae1f805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:26.894827 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932e4e8-c1fb-4bef-b74d-6317bae1f805 admin admin] PATH_INFO: `/v3/projects/2dfdab2067934acdb274b9314bfdb3f7` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:26.896062 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3932e4e8-c1fb-4bef-b74d-6317bae1f805 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dfdab2067934acdb274b9314bfdb3f7)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:26.897971 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3932e4e8-c1fb-4bef-b74d-6317bae1f805 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:26.998938 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3932e4e8-c1fb-4bef-b74d-6317bae1f805 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dfdab2067934acdb274b9314bfdb3f7', 'request_id': 'req-3932e4e8-c1fb-4bef-b74d-6317bae1f805'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:27.021640 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3932e4e8-c1fb-4bef-b74d-6317bae1f805 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dfdab2067934acdb274b9314bfdb3f7', 'request_id': 'req-3932e4e8-c1fb-4bef-b74d-6317bae1f805'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:27.051345 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3932e4e8-c1fb-4bef-b74d-6317bae1f805 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dfdab2067934acdb274b9314bfdb3f7', 'request_id': 'req-3932e4e8-c1fb-4bef-b74d-6317bae1f805'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:27.073695 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3932e4e8-c1fb-4bef-b74d-6317bae1f805 admin admin] The token cache is being invalidate because project 2dfdab2067934acdb274b9314bfdb3f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:27:27.074937 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3932e4e8-c1fb-4bef-b74d-6317bae1f805 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:27.079204 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3932e4e8-c1fb-4bef-b74d-6317bae1f805 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:27.082874 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 965/1932] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:27:26 2022] DELETE /identity/v3/projects/2dfdab2067934acdb274b9314bfdb3f7 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:27:27.089300 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80472a34-ed2d-437d-bb4c-29c266dad80c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:27.089593 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80472a34-ed2d-437d-bb4c-29c266dad80c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:27.090295 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80472a34-ed2d-437d-bb4c-29c266dad80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:27.138410 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-80472a34-ed2d-437d-bb4c-29c266dad80c None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:27:27.144656 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-80472a34-ed2d-437d-bb4c-29c266dad80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:27.298601 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 968/1933] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:27:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:27:27.312402 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9d35c-2c61-48cf-af3d-7ff502b32f11 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:27.312402 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9d35c-2c61-48cf-af3d-7ff502b32f11 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:27.312402 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9d35c-2c61-48cf-af3d-7ff502b32f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:27.340957 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-25c9d35c-2c61-48cf-af3d-7ff502b32f11 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:27:27.343708 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-25c9d35c-2c61-48cf-af3d-7ff502b32f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:27.354519 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 966/1934] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:27:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:27:27.358801 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80472a34-ed2d-437d-bb4c-29c266dad80c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:27.370384 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80472a34-ed2d-437d-bb4c-29c266dad80c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:27.414315 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589aedca-6ef8-48c3-8d38-e014d3f95b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:27.415106 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589aedca-6ef8-48c3-8d38-e014d3f95b12 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:27.415536 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589aedca-6ef8-48c3-8d38-e014d3f95b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:27.415993 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589aedca-6ef8-48c3-8d38-e014d3f95b12 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:27.416957 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589aedca-6ef8-48c3-8d38-e014d3f95b12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:27:27.417404 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589aedca-6ef8-48c3-8d38-e014d3f95b12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:27.419190 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589aedca-6ef8-48c3-8d38-e014d3f95b12 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:27.441033 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 969/1935] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:27:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:27:27.445524 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c9d35c-2c61-48cf-af3d-7ff502b32f11 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:27.459514 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c9d35c-2c61-48cf-af3d-7ff502b32f11 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:27.471718 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f794310-3b52-4384-bf21-6997727b6b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:27.472374 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f794310-3b52-4384-bf21-6997727b6b10 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:27.472658 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f794310-3b52-4384-bf21-6997727b6b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:27.472930 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f794310-3b52-4384-bf21-6997727b6b10 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:27.473622 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f794310-3b52-4384-bf21-6997727b6b10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:27.475590 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f794310-3b52-4384-bf21-6997727b6b10 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:27.527962 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 967/1936] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:27:27 2022] POST /identity/v3/projects => generated 355 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:27:27.533290 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589aedca-6ef8-48c3-8d38-e014d3f95b12 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:27.547769 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589aedca-6ef8-48c3-8d38-e014d3f95b12 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:27.631895 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df204e63-bedc-4505-ae12-bd6dd902ab36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:27.632693 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df204e63-bedc-4505-ae12-bd6dd902ab36 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:27.632980 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df204e63-bedc-4505-ae12-bd6dd902ab36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:27.633377 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df204e63-bedc-4505-ae12-bd6dd902ab36 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:27.634209 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df204e63-bedc-4505-ae12-bd6dd902ab36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:27.636371 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df204e63-bedc-4505-ae12-bd6dd902ab36 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:27.678950 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 970/1937] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:27:27 2022] POST /identity/v3/users => generated 278 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:27:27.681982 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f794310-3b52-4384-bf21-6997727b6b10 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:27.693547 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f794310-3b52-4384-bf21-6997727b6b10 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:27.700296 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed6934a-a782-4ff7-884d-c677bb18806a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:27.700943 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed6934a-a782-4ff7-884d-c677bb18806a admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:27.701720 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed6934a-a782-4ff7-884d-c677bb18806a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:27.702006 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed6934a-a782-4ff7-884d-c677bb18806a admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:27.702715 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed6934a-a782-4ff7-884d-c677bb18806a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:27.705001 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed6934a-a782-4ff7-884d-c677bb18806a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:27.719266 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 968/1938] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:27:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 10:27:27.724153 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df204e63-bedc-4505-ae12-bd6dd902ab36 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:27.737924 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df204e63-bedc-4505-ae12-bd6dd902ab36 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:27.744515 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0ab84c-a435-45d1-a3f6-9e59554667e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:27.744721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ab84c-a435-45d1-a3f6-9e59554667e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:27.745001 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ab84c-a435-45d1-a3f6-9e59554667e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:27.745344 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ab84c-a435-45d1-a3f6-9e59554667e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:27.746324 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0ab84c-a435-45d1-a3f6-9e59554667e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:27.748284 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0ab84c-a435-45d1-a3f6-9e59554667e0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:27.763906 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 971/1939] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:27:27 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:27:27.769795 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed6934a-a782-4ff7-884d-c677bb18806a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:27.793994 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed6934a-a782-4ff7-884d-c677bb18806a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:27.806227 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba114a25-cf46-4053-814a-5b5fc7ec43a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:27.807096 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba114a25-cf46-4053-814a-5b5fc7ec43a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:27.807384 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba114a25-cf46-4053-814a-5b5fc7ec43a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:27.807663 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba114a25-cf46-4053-814a-5b5fc7ec43a0 admin admin] PATH_INFO: `/v3/projects/6e04382dad9445f8a366d204a4afda88/users/201381677a0c4f43957c38a47a821b90/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:27.812031 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:27:27.846775 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba114a25-cf46-4053-814a-5b5fc7ec43a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e04382dad9445f8a366d204a4afda88, user_id=201381677a0c4f43957c38a47a821b90, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:27.849931 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba114a25-cf46-4053-814a-5b5fc7ec43a0 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:27.880833 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 969/1940] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:27:27 2022] PUT /identity/v3/projects/6e04382dad9445f8a366d204a4afda88/users/201381677a0c4f43957c38a47a821b90/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:27:27.892001 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d538e103-f33b-4251-96ab-23479aefcc7f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:27.892317 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d538e103-f33b-4251-96ab-23479aefcc7f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:27.892601 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d538e103-f33b-4251-96ab-23479aefcc7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:27.956187 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d538e103-f33b-4251-96ab-23479aefcc7f None None] MFA Rules not processed for user `201381677a0c4f43957c38a47a821b90`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:27:27.959261 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d538e103-f33b-4251-96ab-23479aefcc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:28.074946 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:27:28.095381 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba114a25-cf46-4053-814a-5b5fc7ec43a0 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:28.097436 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 972/1941] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:27:27 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:27:28.223029 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba114a25-cf46-4053-814a-5b5fc7ec43a0 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:28.229864 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252d5aa4-b1f9-46f7-869f-74aa9d6eacd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:28.251635 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d5aa4-b1f9-46f7-869f-74aa9d6eacd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:28.252348 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d5aa4-b1f9-46f7-869f-74aa9d6eacd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:28.254023 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d5aa4-b1f9-46f7-869f-74aa9d6eacd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:28.254829 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252d5aa4-b1f9-46f7-869f-74aa9d6eacd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:28.257245 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252d5aa4-b1f9-46f7-869f-74aa9d6eacd5 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:28.319399 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 970/1942] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:27:28 2022] POST /identity/v3/projects => generated 355 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:27:28.323474 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d538e103-f33b-4251-96ab-23479aefcc7f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:28.335117 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d538e103-f33b-4251-96ab-23479aefcc7f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:28.402806 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3b3291-e1f3-4d5c-856e-e6b30a4e7099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:28.403411 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3b3291-e1f3-4d5c-856e-e6b30a4e7099 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:28.403691 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3b3291-e1f3-4d5c-856e-e6b30a4e7099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:28.403970 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3b3291-e1f3-4d5c-856e-e6b30a4e7099 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:28.404669 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3b3291-e1f3-4d5c-856e-e6b30a4e7099 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:28.412523 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3b3291-e1f3-4d5c-856e-e6b30a4e7099 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:28.459198 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 973/1943] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:27:28 2022] POST /identity/v3/users => generated 284 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:27:28.463371 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252d5aa4-b1f9-46f7-869f-74aa9d6eacd5 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:28.474690 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252d5aa4-b1f9-46f7-869f-74aa9d6eacd5 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:28.488628 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf9d5cb-cf8f-4ad4-bfda-0c1df153a632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:28.489531 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf9d5cb-cf8f-4ad4-bfda-0c1df153a632 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:28.489951 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf9d5cb-cf8f-4ad4-bfda-0c1df153a632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:28.490364 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf9d5cb-cf8f-4ad4-bfda-0c1df153a632 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:28.491643 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf9d5cb-cf8f-4ad4-bfda-0c1df153a632 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:28.496233 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf9d5cb-cf8f-4ad4-bfda-0c1df153a632 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:28.536313 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 971/1944] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:27:28.536920 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3b3291-e1f3-4d5c-856e-e6b30a4e7099 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:28.550441 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3b3291-e1f3-4d5c-856e-e6b30a4e7099 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:28.555948 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da92a5c4-0731-4d6f-8963-5d46a10db0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:28.556777 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da92a5c4-0731-4d6f-8963-5d46a10db0b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:28.556993 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da92a5c4-0731-4d6f-8963-5d46a10db0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:28.557320 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da92a5c4-0731-4d6f-8963-5d46a10db0b0 admin admin] PATH_INFO: `/v3/projects/f051cb0057c14a5891a0b808b1d15d74/users/b02c6ed884504d92b3040cf7e2a4f4fc/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:28.573814 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da92a5c4-0731-4d6f-8963-5d46a10db0b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f051cb0057c14a5891a0b808b1d15d74, user_id=b02c6ed884504d92b3040cf7e2a4f4fc, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:28.577053 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da92a5c4-0731-4d6f-8963-5d46a10db0b0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:28.599331 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 974/1945] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:27:28 2022] PUT /identity/v3/projects/f051cb0057c14a5891a0b808b1d15d74/users/b02c6ed884504d92b3040cf7e2a4f4fc/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:27:28.603720 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf9d5cb-cf8f-4ad4-bfda-0c1df153a632 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:28.620845 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf9d5cb-cf8f-4ad4-bfda-0c1df153a632 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:28.733393 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79536bed-d9ee-4ad5-a529-6c64606ce4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:28.746351 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79536bed-d9ee-4ad5-a529-6c64606ce4fa admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:28.747479 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79536bed-d9ee-4ad5-a529-6c64606ce4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:28.747802 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79536bed-d9ee-4ad5-a529-6c64606ce4fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:28.750367 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79536bed-d9ee-4ad5-a529-6c64606ce4fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:28.754750 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79536bed-d9ee-4ad5-a529-6c64606ce4fa admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:28.776878 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 972/1946] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:27:28.777086 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da92a5c4-0731-4d6f-8963-5d46a10db0b0 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:28.789411 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da92a5c4-0731-4d6f-8963-5d46a10db0b0 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:28.795126 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed4eee7-09ee-4cdf-97b4-9245edecf222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:28.795968 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed4eee7-09ee-4cdf-97b4-9245edecf222 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:28.796297 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed4eee7-09ee-4cdf-97b4-9245edecf222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:28.796633 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed4eee7-09ee-4cdf-97b4-9245edecf222 admin admin] PATH_INFO: `/v3/projects/f051cb0057c14a5891a0b808b1d15d74/users/b02c6ed884504d92b3040cf7e2a4f4fc/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:28.798836 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed4eee7-09ee-4cdf-97b4-9245edecf222 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f051cb0057c14a5891a0b808b1d15d74, user_id=b02c6ed884504d92b3040cf7e2a4f4fc, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:28.801639 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed4eee7-09ee-4cdf-97b4-9245edecf222 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:28.825892 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad50c2-7bd7-4ee7-b71e-c615700de118 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:28.826179 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad50c2-7bd7-4ee7-b71e-c615700de118 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:28.826457 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad50c2-7bd7-4ee7-b71e-c615700de118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:28.828868 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 975/1947] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:27:28 2022] PUT /identity/v3/projects/f051cb0057c14a5891a0b808b1d15d74/users/b02c6ed884504d92b3040cf7e2a4f4fc/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:27:28.859027 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c1ad50c2-7bd7-4ee7-b71e-c615700de118 None None] MFA Rules not processed for user `b02c6ed884504d92b3040cf7e2a4f4fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:27:28.860261 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c1ad50c2-7bd7-4ee7-b71e-c615700de118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:28.954460 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:27:29.036089 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 973/1948] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:27:28 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:27:29.037155 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed4eee7-09ee-4cdf-97b4-9245edecf222 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:29.039393 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3ed4eee7-09ee-4cdf-97b4-9245edecf222 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:29.204631 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed4eee7-09ee-4cdf-97b4-9245edecf222 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:29.213395 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c256403b-ed4a-4ee2-8c88-fab9922696d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:29.216435 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256403b-ed4a-4ee2-8c88-fab9922696d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:29.216843 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256403b-ed4a-4ee2-8c88-fab9922696d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:29.217211 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256403b-ed4a-4ee2-8c88-fab9922696d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:29.237808 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c256403b-ed4a-4ee2-8c88-fab9922696d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:29.239945 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c256403b-ed4a-4ee2-8c88-fab9922696d7 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:29.280465 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 976/1949] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:27:29 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:27:29.440387 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ad50c2-7bd7-4ee7-b71e-c615700de118 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:29.460526 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ad50c2-7bd7-4ee7-b71e-c615700de118 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:29.468674 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f5a8e3-2a8c-4ef2-8ef3-c527cefb0672 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:29.469760 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f5a8e3-2a8c-4ef2-8ef3-c527cefb0672 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:29.470257 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f5a8e3-2a8c-4ef2-8ef3-c527cefb0672 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:29.470797 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f5a8e3-2a8c-4ef2-8ef3-c527cefb0672 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:29.562793 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f5a8e3-2a8c-4ef2-8ef3-c527cefb0672 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:29.564675 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f5a8e3-2a8c-4ef2-8ef3-c527cefb0672 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:29.579944 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 974/1950] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:27:29 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:27:30.460561 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c76aea-12d5-40d2-98fc-517d1d4af278 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:30.461095 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c76aea-12d5-40d2-98fc-517d1d4af278 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:30.462058 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c76aea-12d5-40d2-98fc-517d1d4af278 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:30.464151 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 977/1951] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 5 10:27:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:27:30.517269 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becec760-4b97-43a8-89d4-98b32b3fbc3b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:30.517805 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becec760-4b97-43a8-89d4-98b32b3fbc3b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:30.517883 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becec760-4b97-43a8-89d4-98b32b3fbc3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:30.539020 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-becec760-4b97-43a8-89d4-98b32b3fbc3b None None] MFA Rules not processed for user `8743f5bb0be54d57a957617636b0b8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:27:30.540589 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-becec760-4b97-43a8-89d4-98b32b3fbc3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:30.623993 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 975/1952] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 5 10:27:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:27:30.665716 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c76aea-12d5-40d2-98fc-517d1d4af278 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:30.667969 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-21c76aea-12d5-40d2-98fc-517d1d4af278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:30.699121 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c76aea-12d5-40d2-98fc-517d1d4af278 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:30.708340 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e641c993-2f5e-482c-933f-196564206bf3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8743f5bb0be54d57a957617636b0b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:30.709771 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e641c993-2f5e-482c-933f-196564206bf3 service placement] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:30.710430 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e641c993-2f5e-482c-933f-196564206bf3 service placement] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:30.710979 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e641c993-2f5e-482c-933f-196564206bf3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:30.739335 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e641c993-2f5e-482c-933f-196564206bf3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:30.742377 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e641c993-2f5e-482c-933f-196564206bf3 service placement] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:30.760272 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 978/1953] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:27:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:27:32.989505 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becec760-4b97-43a8-89d4-98b32b3fbc3b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:32.991109 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-becec760-4b97-43a8-89d4-98b32b3fbc3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:33.127147 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becec760-4b97-43a8-89d4-98b32b3fbc3b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:33.136718 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65573747-6cd6-44ae-bc28-6e5e5575f228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:33.137549 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65573747-6cd6-44ae-bc28-6e5e5575f228 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:33.137838 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65573747-6cd6-44ae-bc28-6e5e5575f228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:33.138596 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65573747-6cd6-44ae-bc28-6e5e5575f228 admin admin] PATH_INFO: `/v3/users/33277dcb6bdf4e1b99afbc28ea2f68c3` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:33.139885 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65573747-6cd6-44ae-bc28-6e5e5575f228 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33277dcb6bdf4e1b99afbc28ea2f68c3)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:33.143380 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65573747-6cd6-44ae-bc28-6e5e5575f228 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:33.181052 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:27:33.270571 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-65573747-6cd6-44ae-bc28-6e5e5575f228 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33277dcb6bdf4e1b99afbc28ea2f68c3'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:33.295438 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-65573747-6cd6-44ae-bc28-6e5e5575f228 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33277dcb6bdf4e1b99afbc28ea2f68c3'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:33.296309 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-65573747-6cd6-44ae-bc28-6e5e5575f228 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33277dcb6bdf4e1b99afbc28ea2f68c3'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:33.317858 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-65573747-6cd6-44ae-bc28-6e5e5575f228 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33277dcb6bdf4e1b99afbc28ea2f68c3'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:33.318708 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-65573747-6cd6-44ae-bc28-6e5e5575f228 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33277dcb6bdf4e1b99afbc28ea2f68c3'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:33.360362 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 976/1954] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:27:32 2022] DELETE /identity/v3/users/33277dcb6bdf4e1b99afbc28ea2f68c3 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:27:33.849927 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e641c993-2f5e-482c-933f-196564206bf3 service placement] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:33.866767 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e641c993-2f5e-482c-933f-196564206bf3 service placement] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:34.006548 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:27:34.065506 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ab8407-26d6-4000-bb0a-332a93a3a2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:34.066146 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ab8407-26d6-4000-bb0a-332a93a3a2d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:34.066430 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ab8407-26d6-4000-bb0a-332a93a3a2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:34.066701 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ab8407-26d6-4000-bb0a-332a93a3a2d3 admin admin] PATH_INFO: `/v3/projects/3186acf85ed345deb6505112ff6f1909` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:34.067909 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ab8407-26d6-4000-bb0a-332a93a3a2d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3186acf85ed345deb6505112ff6f1909)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:34.076797 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ab8407-26d6-4000-bb0a-332a93a3a2d3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:34.190298 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-84ab8407-26d6-4000-bb0a-332a93a3a2d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3186acf85ed345deb6505112ff6f1909', 'request_id': 'req-84ab8407-26d6-4000-bb0a-332a93a3a2d3'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:34.211672 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-84ab8407-26d6-4000-bb0a-332a93a3a2d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3186acf85ed345deb6505112ff6f1909', 'request_id': 'req-84ab8407-26d6-4000-bb0a-332a93a3a2d3'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:34.220378 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-84ab8407-26d6-4000-bb0a-332a93a3a2d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3186acf85ed345deb6505112ff6f1909', 'request_id': 'req-84ab8407-26d6-4000-bb0a-332a93a3a2d3'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:34.235703 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-84ab8407-26d6-4000-bb0a-332a93a3a2d3 admin admin] The token cache is being invalidate because project 3186acf85ed345deb6505112ff6f1909 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:27:34.235930 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-84ab8407-26d6-4000-bb0a-332a93a3a2d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:34.236820 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-84ab8407-26d6-4000-bb0a-332a93a3a2d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:34.242004 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 979/1955] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:27:33 2022] DELETE /identity/v3/projects/3186acf85ed345deb6505112ff6f1909 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:27:34.243390 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65573747-6cd6-44ae-bc28-6e5e5575f228 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:34.244867 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-65573747-6cd6-44ae-bc28-6e5e5575f228 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:34.268432 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65573747-6cd6-44ae-bc28-6e5e5575f228 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:34.397472 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d97dac-d347-46b1-a399-ba4b0baf7f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:34.398210 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d97dac-d347-46b1-a399-ba4b0baf7f9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:34.398563 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d97dac-d347-46b1-a399-ba4b0baf7f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:34.398952 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d97dac-d347-46b1-a399-ba4b0baf7f9a admin admin] PATH_INFO: `/v3/users/3ec89ea8841d414aad41891f310538bd` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:34.400292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d97dac-d347-46b1-a399-ba4b0baf7f9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ec89ea8841d414aad41891f310538bd)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:34.402602 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d97dac-d347-46b1-a399-ba4b0baf7f9a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:34.492530 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-75d97dac-d347-46b1-a399-ba4b0baf7f9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ec89ea8841d414aad41891f310538bd'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:34.512717 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-75d97dac-d347-46b1-a399-ba4b0baf7f9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ec89ea8841d414aad41891f310538bd'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:34.514101 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-75d97dac-d347-46b1-a399-ba4b0baf7f9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ec89ea8841d414aad41891f310538bd'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:34.535298 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-75d97dac-d347-46b1-a399-ba4b0baf7f9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ec89ea8841d414aad41891f310538bd'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:34.536436 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-75d97dac-d347-46b1-a399-ba4b0baf7f9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ec89ea8841d414aad41891f310538bd'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:34.554366 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 977/1956] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:27:34 2022] DELETE /identity/v3/users/3ec89ea8841d414aad41891f310538bd => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:27:34.901780 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a44ac25-62de-450f-bf40-0806f662c962 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:34.902303 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a44ac25-62de-450f-bf40-0806f662c962 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:34.902303 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a44ac25-62de-450f-bf40-0806f662c962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:34.934693 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8a44ac25-62de-450f-bf40-0806f662c962 None None] MFA Rules not processed for user `9697af06b1104425aed6692ba4f29238`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:27:34.936042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8a44ac25-62de-450f-bf40-0806f662c962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:35.044579 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 980/1957] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:27:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:27:35.049875 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d97dac-d347-46b1-a399-ba4b0baf7f9a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:35.051933 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-75d97dac-d347-46b1-a399-ba4b0baf7f9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:35.083113 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:27:35.091689 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a44ac25-62de-450f-bf40-0806f662c962 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:35.094191 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8a44ac25-62de-450f-bf40-0806f662c962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:35.209821 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:27:35.216020 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a44ac25-62de-450f-bf40-0806f662c962 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:35.222131 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ae246-9255-48a7-93e8-7e113ea58ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:35.222553 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d97dac-d347-46b1-a399-ba4b0baf7f9a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:35.223350 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ae246-9255-48a7-93e8-7e113ea58ee9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:35.223887 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ae246-9255-48a7-93e8-7e113ea58ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:35.224322 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ae246-9255-48a7-93e8-7e113ea58ee9 admin admin] PATH_INFO: `/v3/projects/b636afcc425c432a9ca0ab3c41667ef7` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:35.226302 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ae246-9255-48a7-93e8-7e113ea58ee9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b636afcc425c432a9ca0ab3c41667ef7)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:35.229228 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ae246-9255-48a7-93e8-7e113ea58ee9 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:35.234695 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f50a8f-6600-4c1d-a25e-fe6d4ffdf2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:35.235434 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f50a8f-6600-4c1d-a25e-fe6d4ffdf2b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:35.236090 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f50a8f-6600-4c1d-a25e-fe6d4ffdf2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:35.236422 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f50a8f-6600-4c1d-a25e-fe6d4ffdf2b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:35.237162 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f50a8f-6600-4c1d-a25e-fe6d4ffdf2b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:35.240268 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f50a8f-6600-4c1d-a25e-fe6d4ffdf2b9 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:35.291001 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 978/1958] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:27:35 2022] POST /identity/v3/projects => generated 385 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:27:35.293100 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f50a8f-6600-4c1d-a25e-fe6d4ffdf2b9 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:35.365831 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-837ae246-9255-48a7-93e8-7e113ea58ee9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b636afcc425c432a9ca0ab3c41667ef7', 'request_id': 'req-837ae246-9255-48a7-93e8-7e113ea58ee9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:35.379222 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-837ae246-9255-48a7-93e8-7e113ea58ee9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b636afcc425c432a9ca0ab3c41667ef7', 'request_id': 'req-837ae246-9255-48a7-93e8-7e113ea58ee9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:35.392335 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-837ae246-9255-48a7-93e8-7e113ea58ee9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b636afcc425c432a9ca0ab3c41667ef7', 'request_id': 'req-837ae246-9255-48a7-93e8-7e113ea58ee9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:35.403471 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f50a8f-6600-4c1d-a25e-fe6d4ffdf2b9 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:35.422045 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-837ae246-9255-48a7-93e8-7e113ea58ee9 admin admin] The token cache is being invalidate because project b636afcc425c432a9ca0ab3c41667ef7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:27:35.422359 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-837ae246-9255-48a7-93e8-7e113ea58ee9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:35.424735 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-837ae246-9255-48a7-93e8-7e113ea58ee9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:35.440275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 981/1959] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:27:35 2022] DELETE /identity/v3/projects/b636afcc425c432a9ca0ab3c41667ef7 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:27:35.441648 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939dc67-e6a9-4df6-89ef-307174f768dd None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:35.442268 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939dc67-e6a9-4df6-89ef-307174f768dd None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:35.442532 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939dc67-e6a9-4df6-89ef-307174f768dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:35.473537 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6939dc67-e6a9-4df6-89ef-307174f768dd None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:27:35.477865 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6939dc67-e6a9-4df6-89ef-307174f768dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:35.541125 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114bd373-ceb8-47ee-ba15-dc981e0446a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:35.541369 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114bd373-ceb8-47ee-ba15-dc981e0446a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:35.541594 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114bd373-ceb8-47ee-ba15-dc981e0446a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:35.541871 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114bd373-ceb8-47ee-ba15-dc981e0446a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:35.542549 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114bd373-ceb8-47ee-ba15-dc981e0446a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:35.549886 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114bd373-ceb8-47ee-ba15-dc981e0446a1 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:35.608359 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 979/1960] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:27:35 2022] POST /identity/v3/users => generated 299 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:27:35.616181 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114bd373-ceb8-47ee-ba15-dc981e0446a1 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:35.619559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-114bd373-ceb8-47ee-ba15-dc981e0446a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:35.633134 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114bd373-ceb8-47ee-ba15-dc981e0446a1 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:35.638149 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe1dba-87cc-4f5a-b02b-31283f569c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:35.638787 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe1dba-87cc-4f5a-b02b-31283f569c76 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:35.639123 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe1dba-87cc-4f5a-b02b-31283f569c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:35.639401 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe1dba-87cc-4f5a-b02b-31283f569c76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:35.640109 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbe1dba-87cc-4f5a-b02b-31283f569c76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:35.641727 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbe1dba-87cc-4f5a-b02b-31283f569c76 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:35.650984 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 982/1961] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:27:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:27:35.656667 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5cc739-6ec8-4c78-81cc-0748a8015cf4 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:35.656952 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5cc739-6ec8-4c78-81cc-0748a8015cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:35.657226 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5cc739-6ec8-4c78-81cc-0748a8015cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:35.664598 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 980/1962] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:27:35 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 10:27:35.670337 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe1dba-87cc-4f5a-b02b-31283f569c76 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:35.687470 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe1dba-87cc-4f5a-b02b-31283f569c76 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:35.688278 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-bb5cc739-6ec8-4c78-81cc-0748a8015cf4 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:27:35.689635 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bb5cc739-6ec8-4c78-81cc-0748a8015cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:35.693868 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee50714-d05f-4646-a149-6e1b9b238ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:35.694851 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee50714-d05f-4646-a149-6e1b9b238ad2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:35.695279 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee50714-d05f-4646-a149-6e1b9b238ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:35.695631 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee50714-d05f-4646-a149-6e1b9b238ad2 admin admin] PATH_INFO: `/v3/projects/7a197ca65c7a440395ab38c607d766f1/users/3a93ce5d2392424f80afe18980c8a37e/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:35.698948 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 983/1963] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:27:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:27:35.703373 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5cc739-6ec8-4c78-81cc-0748a8015cf4 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:35.719476 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee50714-d05f-4646-a149-6e1b9b238ad2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a197ca65c7a440395ab38c607d766f1, user_id=3a93ce5d2392424f80afe18980c8a37e, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:35.722251 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5cc739-6ec8-4c78-81cc-0748a8015cf4 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:35.722504 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee50714-d05f-4646-a149-6e1b9b238ad2 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:35.735274 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:27:35.741140 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82777d3-fa3f-495a-ae70-be50a055507d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:35.742054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82777d3-fa3f-495a-ae70-be50a055507d admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:35.742493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82777d3-fa3f-495a-ae70-be50a055507d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:35.743424 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82777d3-fa3f-495a-ae70-be50a055507d admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:35.744341 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82777d3-fa3f-495a-ae70-be50a055507d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:27:35.744643 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82777d3-fa3f-495a-ae70-be50a055507d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:35.746474 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82777d3-fa3f-495a-ae70-be50a055507d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:35.749665 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 981/1964] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:27:35 2022] PUT /identity/v3/projects/7a197ca65c7a440395ab38c607d766f1/users/3a93ce5d2392424f80afe18980c8a37e/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:27:35.753682 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee50714-d05f-4646-a149-6e1b9b238ad2 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:35.784507 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 984/1965] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:27:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:27:35.786928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82777d3-fa3f-495a-ae70-be50a055507d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:35.798812 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82777d3-fa3f-495a-ae70-be50a055507d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:35.913509 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eec827-e771-440a-b9b1-56057abc7e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:35.914151 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eec827-e771-440a-b9b1-56057abc7e26 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:35.914634 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eec827-e771-440a-b9b1-56057abc7e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:35.915096 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eec827-e771-440a-b9b1-56057abc7e26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:35.915918 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eec827-e771-440a-b9b1-56057abc7e26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:35.918081 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eec827-e771-440a-b9b1-56057abc7e26 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:35.921936 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee50714-d05f-4646-a149-6e1b9b238ad2 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:35.935227 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd4756-1e76-4e62-a529-5e92a713a358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:35.947899 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd4756-1e76-4e62-a529-5e92a713a358 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:35.947899 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd4756-1e76-4e62-a529-5e92a713a358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:35.947899 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd4756-1e76-4e62-a529-5e92a713a358 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:35.947899 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfd4756-1e76-4e62-a529-5e92a713a358 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:35.947899 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfd4756-1e76-4e62-a529-5e92a713a358 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:35.948344 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:27:35.965334 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 982/1966] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:27:35 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:27:35.967703 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd4756-1e76-4e62-a529-5e92a713a358 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:35.975571 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 985/1967] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:27:35 2022] POST /identity/v3/projects => generated 385 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:27:35.980214 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eec827-e771-440a-b9b1-56057abc7e26 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:35.992004 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd4756-1e76-4e62-a529-5e92a713a358 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:36.131747 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eec827-e771-440a-b9b1-56057abc7e26 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:36.140987 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c85e4-4571-42cf-af2c-9f9baf937b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:36.142045 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c85e4-4571-42cf-af2c-9f9baf937b8c admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:36.142670 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c85e4-4571-42cf-af2c-9f9baf937b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:36.143152 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c85e4-4571-42cf-af2c-9f9baf937b8c admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:36.144631 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205c85e4-4571-42cf-af2c-9f9baf937b8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:36.148381 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205c85e4-4571-42cf-af2c-9f9baf937b8c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:36.163378 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b256c-ad77-46b9-a04e-630a233c56a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:36.164386 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b256c-ad77-46b9-a04e-630a233c56a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:36.164977 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b256c-ad77-46b9-a04e-630a233c56a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:36.165167 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b256c-ad77-46b9-a04e-630a233c56a5 admin admin] PATH_INFO: `/v3/projects/7a197ca65c7a440395ab38c607d766f1/users/3a93ce5d2392424f80afe18980c8a37e/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:36.173021 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212b256c-ad77-46b9-a04e-630a233c56a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a197ca65c7a440395ab38c607d766f1, user_id=3a93ce5d2392424f80afe18980c8a37e, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:36.176665 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212b256c-ad77-46b9-a04e-630a233c56a5 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:36.201693 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 986/1968] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:27:35 2022] POST /identity/v3/users => generated 293 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:27:36.206377 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c85e4-4571-42cf-af2c-9f9baf937b8c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:36.217252 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 983/1969] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:27:35 2022] PUT /identity/v3/projects/7a197ca65c7a440395ab38c607d766f1/users/3a93ce5d2392424f80afe18980c8a37e/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:27:36.234099 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c85e4-4571-42cf-af2c-9f9baf937b8c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:36.359059 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af31795-f8b0-4912-8f8c-cbf891468aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:36.361265 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af31795-f8b0-4912-8f8c-cbf891468aa7 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:36.361818 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af31795-f8b0-4912-8f8c-cbf891468aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:36.362398 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af31795-f8b0-4912-8f8c-cbf891468aa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:36.363102 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af31795-f8b0-4912-8f8c-cbf891468aa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:36.366332 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af31795-f8b0-4912-8f8c-cbf891468aa7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:36.384018 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 987/1970] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:27:36 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:27:36.387639 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b256c-ad77-46b9-a04e-630a233c56a5 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:36.401890 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b256c-ad77-46b9-a04e-630a233c56a5 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:36.411105 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8880a333-e913-46c4-9ee2-06ac62ee406b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:36.411791 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8880a333-e913-46c4-9ee2-06ac62ee406b admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:36.412073 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8880a333-e913-46c4-9ee2-06ac62ee406b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:36.412775 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8880a333-e913-46c4-9ee2-06ac62ee406b admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:36.413032 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8880a333-e913-46c4-9ee2-06ac62ee406b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:36.414696 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8880a333-e913-46c4-9ee2-06ac62ee406b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:36.430982 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 984/1971] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:27:36 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 10:27:36.431671 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af31795-f8b0-4912-8f8c-cbf891468aa7 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:36.446121 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af31795-f8b0-4912-8f8c-cbf891468aa7 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:36.454638 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075581c4-6631-4010-a30b-aa02c2ba8e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:36.455026 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075581c4-6631-4010-a30b-aa02c2ba8e28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:36.455361 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075581c4-6631-4010-a30b-aa02c2ba8e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:36.455587 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075581c4-6631-4010-a30b-aa02c2ba8e28 admin admin] PATH_INFO: `/v3/projects/130d2fa7424b4030bffa7f924cb1191e/users/090c86a769c04fd4863081a9d513b519/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:36.486238 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075581c4-6631-4010-a30b-aa02c2ba8e28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=130d2fa7424b4030bffa7f924cb1191e, user_id=090c86a769c04fd4863081a9d513b519, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:36.506806 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075581c4-6631-4010-a30b-aa02c2ba8e28 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:36.539632 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 988/1972] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:27:36 2022] PUT /identity/v3/projects/130d2fa7424b4030bffa7f924cb1191e/users/090c86a769c04fd4863081a9d513b519/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:27:36.550701 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ee335-bb4a-4cdc-8eca-c282f97a4cad None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:36.551022 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ee335-bb4a-4cdc-8eca-c282f97a4cad None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:36.551293 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ee335-bb4a-4cdc-8eca-c282f97a4cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:36.601847 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-042ee335-bb4a-4cdc-8eca-c282f97a4cad None None] MFA Rules not processed for user `090c86a769c04fd4863081a9d513b519`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:27:36.602439 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-042ee335-bb4a-4cdc-8eca-c282f97a4cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:36.709930 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 985/1973] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:27:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:27:36.718290 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075581c4-6631-4010-a30b-aa02c2ba8e28 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:36.720176 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-075581c4-6631-4010-a30b-aa02c2ba8e28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:36.828412 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075581c4-6631-4010-a30b-aa02c2ba8e28 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:36.833630 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e367d8-421c-432b-990f-e77575ffc4d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:36.834606 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e367d8-421c-432b-990f-e77575ffc4d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:36.835117 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e367d8-421c-432b-990f-e77575ffc4d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:36.835580 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e367d8-421c-432b-990f-e77575ffc4d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:36.859209 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e367d8-421c-432b-990f-e77575ffc4d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:36.860966 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e367d8-421c-432b-990f-e77575ffc4d2 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:36.865087 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:27:36.874482 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 989/1974] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:27:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:27:42.501175 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f6c98d-a831-4600-b932-aa3a0ff66077 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:42.501756 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f6c98d-a831-4600-b932-aa3a0ff66077 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:42.501756 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f6c98d-a831-4600-b932-aa3a0ff66077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:42.515250 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:27:42.538580 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-30f6c98d-a831-4600-b932-aa3a0ff66077 None None] MFA Rules not processed for user `3a93ce5d2392424f80afe18980c8a37e`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:27:42.539986 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-30f6c98d-a831-4600-b932-aa3a0ff66077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:42.614354 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 986/1975] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:27:42 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:27:42.627779 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e367d8-421c-432b-990f-e77575ffc4d2 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:42.629244 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-15e367d8-421c-432b-990f-e77575ffc4d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:42.726025 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e367d8-421c-432b-990f-e77575ffc4d2 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:42.730003 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc62f3d-f1a9-4b9e-ac44-ef62c9481ca0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac0a8cf7f8b74e88a04444ec38547528', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:42.730617 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc62f3d-f1a9-4b9e-ac44-ef62c9481ca0 service glance] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:42.730894 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc62f3d-f1a9-4b9e-ac44-ef62c9481ca0 service glance] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:42.731220 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc62f3d-f1a9-4b9e-ac44-ef62c9481ca0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:42.733107 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2cc62f3d-f1a9-4b9e-ac44-ef62c9481ca0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:42.795121 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc62f3d-f1a9-4b9e-ac44-ef62c9481ca0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:42.796575 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc62f3d-f1a9-4b9e-ac44-ef62c9481ca0 service glance] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:42.808809 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 990/1976] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:27:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:27:42.944538 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a9e2e8-5cf6-4bb1-a30f-5b010a14e37f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:42.944831 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a9e2e8-5cf6-4bb1-a30f-5b010a14e37f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:42.945110 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a9e2e8-5cf6-4bb1-a30f-5b010a14e37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:42.979121 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-f9a9e2e8-5cf6-4bb1-a30f-5b010a14e37f None None] MFA Rules not processed for user `52fc3ba90dc045c2a5b7bb730da96acc`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:27:42.980503 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f9a9e2e8-5cf6-4bb1-a30f-5b010a14e37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:43.055407 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 987/1977] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 5 10:27:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:27:43.108173 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc62f3d-f1a9-4b9e-ac44-ef62c9481ca0 service glance] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:43.109772 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2cc62f3d-f1a9-4b9e-ac44-ef62c9481ca0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:43.185567 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc62f3d-f1a9-4b9e-ac44-ef62c9481ca0 service glance] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:43.190660 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d166ae-523c-4837-a18e-da8b1ddb83db service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c80cf0e429f04f229aea1aa9293a93b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:43.191319 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d166ae-523c-4837-a18e-da8b1ddb83db service swift] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:43.191603 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d166ae-523c-4837-a18e-da8b1ddb83db service swift] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:43.191880 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d166ae-523c-4837-a18e-da8b1ddb83db service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:43.200425 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d166ae-523c-4837-a18e-da8b1ddb83db service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:43.202315 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d166ae-523c-4837-a18e-da8b1ddb83db service swift] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:43.210959 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 991/1978] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 5 10:27:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 10:27:49.835387 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a9e2e8-5cf6-4bb1-a30f-5b010a14e37f None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:50.007194 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a9e2e8-5cf6-4bb1-a30f-5b010a14e37f None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:50.007194 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec0bdf-c459-4eb1-be54-50e7566998bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:50.013982 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec0bdf-c459-4eb1-be54-50e7566998bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:50.013982 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec0bdf-c459-4eb1-be54-50e7566998bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:50.013982 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec0bdf-c459-4eb1-be54-50e7566998bc admin admin] PATH_INFO: `/v3/users/090c86a769c04fd4863081a9d513b519` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:50.013982 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feec0bdf-c459-4eb1-be54-50e7566998bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=090c86a769c04fd4863081a9d513b519)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:50.013982 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feec0bdf-c459-4eb1-be54-50e7566998bc admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:50.155685 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:27:50.177805 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-feec0bdf-c459-4eb1-be54-50e7566998bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '090c86a769c04fd4863081a9d513b519'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:50.195486 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-feec0bdf-c459-4eb1-be54-50e7566998bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '090c86a769c04fd4863081a9d513b519'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:50.196430 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-feec0bdf-c459-4eb1-be54-50e7566998bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '090c86a769c04fd4863081a9d513b519'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:50.217080 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-feec0bdf-c459-4eb1-be54-50e7566998bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '090c86a769c04fd4863081a9d513b519'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:50.217852 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-feec0bdf-c459-4eb1-be54-50e7566998bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '090c86a769c04fd4863081a9d513b519'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:50.275030 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 988/1979] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:27:49 2022] DELETE /identity/v3/users/090c86a769c04fd4863081a9d513b519 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:27:50.280233 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46638e-dd73-4e40-ad3c-0c44b82bd815 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:50.280746 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46638e-dd73-4e40-ad3c-0c44b82bd815 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:50.280916 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46638e-dd73-4e40-ad3c-0c44b82bd815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:50.334006 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-fe46638e-dd73-4e40-ad3c-0c44b82bd815 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:27:50.335412 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fe46638e-dd73-4e40-ad3c-0c44b82bd815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:50.379496 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:27:50.453793 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 992/1980] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:27:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:27:50.465401 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec0bdf-c459-4eb1-be54-50e7566998bc admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:50.466888 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-feec0bdf-c459-4eb1-be54-50e7566998bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:50.558802 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec0bdf-c459-4eb1-be54-50e7566998bc admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:50.575553 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441850a3-6eca-4e22-b5bb-fed9f536b381 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:50.576373 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441850a3-6eca-4e22-b5bb-fed9f536b381 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:50.576666 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441850a3-6eca-4e22-b5bb-fed9f536b381 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:50.576950 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441850a3-6eca-4e22-b5bb-fed9f536b381 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:50.589105 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441850a3-6eca-4e22-b5bb-fed9f536b381 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:50.591257 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441850a3-6eca-4e22-b5bb-fed9f536b381 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:50.604467 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 989/1981] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:27:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:27:51.082849 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe46638e-dd73-4e40-ad3c-0c44b82bd815 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:51.083272 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fe46638e-dd73-4e40-ad3c-0c44b82bd815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:51.130542 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe46638e-dd73-4e40-ad3c-0c44b82bd815 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:51.177763 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d219455-a5e3-4cc6-8a4c-1e010116121a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:51.178570 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d219455-a5e3-4cc6-8a4c-1e010116121a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:51.179664 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d219455-a5e3-4cc6-8a4c-1e010116121a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:51.180833 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d219455-a5e3-4cc6-8a4c-1e010116121a admin admin] PATH_INFO: `/v3/projects/130d2fa7424b4030bffa7f924cb1191e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:51.183744 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d219455-a5e3-4cc6-8a4c-1e010116121a admin admin] RBAC: Authorizing `identity:delete_project(project_id=130d2fa7424b4030bffa7f924cb1191e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:51.192211 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d219455-a5e3-4cc6-8a4c-1e010116121a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:51.389354 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7d219455-a5e3-4cc6-8a4c-1e010116121a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '130d2fa7424b4030bffa7f924cb1191e', 'request_id': 'req-7d219455-a5e3-4cc6-8a4c-1e010116121a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:51.412120 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7d219455-a5e3-4cc6-8a4c-1e010116121a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '130d2fa7424b4030bffa7f924cb1191e', 'request_id': 'req-7d219455-a5e3-4cc6-8a4c-1e010116121a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:51.426083 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7d219455-a5e3-4cc6-8a4c-1e010116121a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '130d2fa7424b4030bffa7f924cb1191e', 'request_id': 'req-7d219455-a5e3-4cc6-8a4c-1e010116121a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:51.457010 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7d219455-a5e3-4cc6-8a4c-1e010116121a admin admin] The token cache is being invalidate because project 130d2fa7424b4030bffa7f924cb1191e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:27:51.457302 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7d219455-a5e3-4cc6-8a4c-1e010116121a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:51.457997 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7d219455-a5e3-4cc6-8a4c-1e010116121a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:51.459811 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 993/1982] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:27:51 2022] DELETE /identity/v3/projects/130d2fa7424b4030bffa7f924cb1191e => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:27:51.464803 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c181e-971f-4d1f-b3ae-25db0108401b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:51.465824 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c181e-971f-4d1f-b3ae-25db0108401b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:51.465824 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c181e-971f-4d1f-b3ae-25db0108401b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:51.508676 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b79c181e-971f-4d1f-b3ae-25db0108401b None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:27:51.509951 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b79c181e-971f-4d1f-b3ae-25db0108401b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:51.669050 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1163e954-f3d0-4535-bf9c-8e9086280e6b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:51.669050 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1163e954-f3d0-4535-bf9c-8e9086280e6b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:51.669353 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1163e954-f3d0-4535-bf9c-8e9086280e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:51.675839 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 990/1983] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:27:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:27:51.707282 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1163e954-f3d0-4535-bf9c-8e9086280e6b None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:27:51.710317 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1163e954-f3d0-4535-bf9c-8e9086280e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:51.741690 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c181e-971f-4d1f-b3ae-25db0108401b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:51.748472 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 994/1984] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:27:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:27:51.754580 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c181e-971f-4d1f-b3ae-25db0108401b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:51.779990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6540181b-923c-40ef-bc87-91bb35360096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:51.782208 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540181b-923c-40ef-bc87-91bb35360096 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:51.786691 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540181b-923c-40ef-bc87-91bb35360096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:51.787385 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540181b-923c-40ef-bc87-91bb35360096 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:51.792218 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6540181b-923c-40ef-bc87-91bb35360096 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:27:51.793777 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6540181b-923c-40ef-bc87-91bb35360096 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:51.796577 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6540181b-923c-40ef-bc87-91bb35360096 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:51.831432 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 991/1985] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:27:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:27:51.832983 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1163e954-f3d0-4535-bf9c-8e9086280e6b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:51.853696 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1163e954-f3d0-4535-bf9c-8e9086280e6b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:51.868059 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c968b7-573b-426b-a23b-be249eb6074e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:51.868652 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c968b7-573b-426b-a23b-be249eb6074e admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:51.869006 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c968b7-573b-426b-a23b-be249eb6074e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:51.870559 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c968b7-573b-426b-a23b-be249eb6074e admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:51.871356 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c968b7-573b-426b-a23b-be249eb6074e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:51.875839 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c968b7-573b-426b-a23b-be249eb6074e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:51.947341 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6540181b-923c-40ef-bc87-91bb35360096 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:51.949954 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 995/1986] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:27:51 2022] POST /identity/v3/projects => generated 375 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:27:51.967376 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6540181b-923c-40ef-bc87-91bb35360096 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:52.076953 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:27:52.113362 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76de7df-580e-4a7e-b9ed-76e3c59e6ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:52.114162 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76de7df-580e-4a7e-b9ed-76e3c59e6ee9 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:52.114518 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76de7df-580e-4a7e-b9ed-76e3c59e6ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:52.114891 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76de7df-580e-4a7e-b9ed-76e3c59e6ee9 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:52.116287 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76de7df-580e-4a7e-b9ed-76e3c59e6ee9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:52.121232 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76de7df-580e-4a7e-b9ed-76e3c59e6ee9 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:52.181422 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c968b7-573b-426b-a23b-be249eb6074e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:52.184351 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-85c968b7-573b-426b-a23b-be249eb6074e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:52.202204 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c968b7-573b-426b-a23b-be249eb6074e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:52.211489 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 992/1987] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:27:51 2022] POST /identity/v3/users => generated 288 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:27:52.212943 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76de7df-580e-4a7e-b9ed-76e3c59e6ee9 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:52.219145 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a037ec-7311-4431-a564-ee11878b8599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:52.220108 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a037ec-7311-4431-a564-ee11878b8599 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:52.220390 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a037ec-7311-4431-a564-ee11878b8599 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:52.222740 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a037ec-7311-4431-a564-ee11878b8599 admin admin] PATH_INFO: `/v3/users/201381677a0c4f43957c38a47a821b90` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:52.225195 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a037ec-7311-4431-a564-ee11878b8599 admin admin] RBAC: Authorizing `identity:delete_user(user_id=201381677a0c4f43957c38a47a821b90)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:52.228447 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a037ec-7311-4431-a564-ee11878b8599 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:52.232646 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76de7df-580e-4a7e-b9ed-76e3c59e6ee9 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:52.246498 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c0dd4-c56f-4cca-9843-50fcd0093588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:52.247211 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c0dd4-c56f-4cca-9843-50fcd0093588 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:52.247501 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c0dd4-c56f-4cca-9843-50fcd0093588 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:52.247777 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c0dd4-c56f-4cca-9843-50fcd0093588 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:52.248474 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97c0dd4-c56f-4cca-9843-50fcd0093588 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:52.250110 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97c0dd4-c56f-4cca-9843-50fcd0093588 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:52.281132 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 993/1988] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:27:52 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:27:52.291288 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c0dd4-c56f-4cca-9843-50fcd0093588 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:52.328948 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c0dd4-c56f-4cca-9843-50fcd0093588 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:52.341484 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f76eff-b320-487d-9500-a6ae72ad4b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:52.342127 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f76eff-b320-487d-9500-a6ae72ad4b69 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:52.342409 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f76eff-b320-487d-9500-a6ae72ad4b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:52.342682 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f76eff-b320-487d-9500-a6ae72ad4b69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:52.343434 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f76eff-b320-487d-9500-a6ae72ad4b69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:52.346266 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f76eff-b320-487d-9500-a6ae72ad4b69 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:52.350810 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-63a037ec-7311-4431-a564-ee11878b8599 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '201381677a0c4f43957c38a47a821b90'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:52.351812 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-63a037ec-7311-4431-a564-ee11878b8599 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '201381677a0c4f43957c38a47a821b90'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:52.352603 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-63a037ec-7311-4431-a564-ee11878b8599 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '201381677a0c4f43957c38a47a821b90'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:52.369193 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 994/1989] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:27:52 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:27:52.370429 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f76eff-b320-487d-9500-a6ae72ad4b69 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:52.372657 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-09f76eff-b320-487d-9500-a6ae72ad4b69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:52.374999 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-63a037ec-7311-4431-a564-ee11878b8599 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '201381677a0c4f43957c38a47a821b90'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:52.386265 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-63a037ec-7311-4431-a564-ee11878b8599 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '201381677a0c4f43957c38a47a821b90'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:52.395571 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=81,keystone:DELETE=16,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:27:52.398684 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f76eff-b320-487d-9500-a6ae72ad4b69 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:52.409933 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f5eef-012c-454c-b477-4d7cbb44d224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:52.411628 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f5eef-012c-454c-b477-4d7cbb44d224 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:52.411988 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f5eef-012c-454c-b477-4d7cbb44d224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:52.412333 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f5eef-012c-454c-b477-4d7cbb44d224 admin admin] PATH_INFO: `/v3/projects/ad5593f1bc6d4f448c2ab09a48ac502e/users/1c7b62b1520a4a0cb430479e279e60d8/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:52.417822 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 996/1990] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:27:52 2022] DELETE /identity/v3/users/201381677a0c4f43957c38a47a821b90 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:27:52.423717 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc892c3a-e808-4d10-a387-bce5510ad624 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:52.424004 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc892c3a-e808-4d10-a387-bce5510ad624 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:52.424432 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc892c3a-e808-4d10-a387-bce5510ad624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:52.451967 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400f5eef-012c-454c-b477-4d7cbb44d224 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad5593f1bc6d4f448c2ab09a48ac502e, user_id=1c7b62b1520a4a0cb430479e279e60d8, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:52.456916 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400f5eef-012c-454c-b477-4d7cbb44d224 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:52.459668 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-bc892c3a-e808-4d10-a387-bce5510ad624 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:27:52.461850 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bc892c3a-e808-4d10-a387-bce5510ad624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:52.488329 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 995/1991] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:27:52 2022] PUT /identity/v3/projects/ad5593f1bc6d4f448c2ab09a48ac502e/users/1c7b62b1520a4a0cb430479e279e60d8/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:27:52.500655 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251c614-c2bc-4123-b25b-4daf23d57c10 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:52.500953 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251c614-c2bc-4123-b25b-4daf23d57c10 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:52.501237 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251c614-c2bc-4123-b25b-4daf23d57c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:52.542168 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 997/1992] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:27:52.548168 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b251c614-c2bc-4123-b25b-4daf23d57c10 None None] MFA Rules not processed for user `1c7b62b1520a4a0cb430479e279e60d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:27:52.549678 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b251c614-c2bc-4123-b25b-4daf23d57c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:52.551539 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc892c3a-e808-4d10-a387-bce5510ad624 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:52.553735 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bc892c3a-e808-4d10-a387-bce5510ad624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:52.655349 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 996/1993] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:27:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:27:52.657243 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc892c3a-e808-4d10-a387-bce5510ad624 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:52.675969 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251c614-c2bc-4123-b25b-4daf23d57c10 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:52.677536 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b251c614-c2bc-4123-b25b-4daf23d57c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:52.730069 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe69edf-2716-489a-a71e-3bba448fa38d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:52.731319 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe69edf-2716-489a-a71e-3bba448fa38d service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:52.731892 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe69edf-2716-489a-a71e-3bba448fa38d service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:52.732369 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe69edf-2716-489a-a71e-3bba448fa38d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:52.767336 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251c614-c2bc-4123-b25b-4daf23d57c10 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:52.791434 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d62baf-5f88-4726-8adb-b50c2d5121cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:52.792095 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d62baf-5f88-4726-8adb-b50c2d5121cf service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:52.793225 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d62baf-5f88-4726-8adb-b50c2d5121cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:52.793225 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d62baf-5f88-4726-8adb-b50c2d5121cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:52.810407 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:27:52.814369 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe69edf-2716-489a-a71e-3bba448fa38d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:52.816753 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe69edf-2716-489a-a71e-3bba448fa38d service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:52.828715 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d62baf-5f88-4726-8adb-b50c2d5121cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:52.830347 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d62baf-5f88-4726-8adb-b50c2d5121cf service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:52.831679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 998/1994] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:27:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:27:52.849643 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 997/1995] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:27:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:27:53.262370 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe69edf-2716-489a-a71e-3bba448fa38d service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:53.264724 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ffe69edf-2716-489a-a71e-3bba448fa38d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:53.297536 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe69edf-2716-489a-a71e-3bba448fa38d service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:53.328423 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eea280-857f-4d89-9b8e-95a94428963d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:53.329110 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eea280-857f-4d89-9b8e-95a94428963d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:53.329399 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eea280-857f-4d89-9b8e-95a94428963d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:53.329675 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eea280-857f-4d89-9b8e-95a94428963d admin admin] PATH_INFO: `/v3/projects/6e04382dad9445f8a366d204a4afda88` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:53.339330 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eea280-857f-4d89-9b8e-95a94428963d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e04382dad9445f8a366d204a4afda88)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:53.341303 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eea280-857f-4d89-9b8e-95a94428963d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:53.481007 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-01eea280-857f-4d89-9b8e-95a94428963d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e04382dad9445f8a366d204a4afda88', 'request_id': 'req-01eea280-857f-4d89-9b8e-95a94428963d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:53.491234 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-01eea280-857f-4d89-9b8e-95a94428963d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e04382dad9445f8a366d204a4afda88', 'request_id': 'req-01eea280-857f-4d89-9b8e-95a94428963d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:53.502107 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-01eea280-857f-4d89-9b8e-95a94428963d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e04382dad9445f8a366d204a4afda88', 'request_id': 'req-01eea280-857f-4d89-9b8e-95a94428963d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:53.517412 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-01eea280-857f-4d89-9b8e-95a94428963d admin admin] The token cache is being invalidate because project 6e04382dad9445f8a366d204a4afda88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:27:53.517730 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-01eea280-857f-4d89-9b8e-95a94428963d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:53.518368 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-01eea280-857f-4d89-9b8e-95a94428963d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:53.520175 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 999/1996] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:27:53 2022] DELETE /identity/v3/projects/6e04382dad9445f8a366d204a4afda88 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:27:53.524168 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d62baf-5f88-4726-8adb-b50c2d5121cf service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:53.536891 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d62baf-5f88-4726-8adb-b50c2d5121cf service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:53.577786 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:27:53.635657 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d581d-5180-4aac-8afe-1cefcea22951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:53.636765 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d581d-5180-4aac-8afe-1cefcea22951 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:53.637108 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d581d-5180-4aac-8afe-1cefcea22951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:53.637442 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d581d-5180-4aac-8afe-1cefcea22951 admin admin] PATH_INFO: `/v3/users/b02c6ed884504d92b3040cf7e2a4f4fc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:53.639296 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52d581d-5180-4aac-8afe-1cefcea22951 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b02c6ed884504d92b3040cf7e2a4f4fc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:53.641379 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52d581d-5180-4aac-8afe-1cefcea22951 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:53.722196 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f52d581d-5180-4aac-8afe-1cefcea22951 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b02c6ed884504d92b3040cf7e2a4f4fc'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:53.735698 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f52d581d-5180-4aac-8afe-1cefcea22951 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b02c6ed884504d92b3040cf7e2a4f4fc'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:53.736396 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f52d581d-5180-4aac-8afe-1cefcea22951 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b02c6ed884504d92b3040cf7e2a4f4fc'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:53.752477 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f52d581d-5180-4aac-8afe-1cefcea22951 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b02c6ed884504d92b3040cf7e2a4f4fc'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:53.754715 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f52d581d-5180-4aac-8afe-1cefcea22951 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b02c6ed884504d92b3040cf7e2a4f4fc'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:53.772510 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 998/1997] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:27:53 2022] DELETE /identity/v3/users/b02c6ed884504d92b3040cf7e2a4f4fc => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:27:54.132111 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eea280-857f-4d89-9b8e-95a94428963d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:54.133603 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-01eea280-857f-4d89-9b8e-95a94428963d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:54.159684 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eea280-857f-4d89-9b8e-95a94428963d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:54.280149 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13372770-9eff-4f45-9708-7c7452469a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:54.281313 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13372770-9eff-4f45-9708-7c7452469a68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:54.281961 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13372770-9eff-4f45-9708-7c7452469a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:54.282422 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13372770-9eff-4f45-9708-7c7452469a68 admin admin] PATH_INFO: `/v3/projects/f051cb0057c14a5891a0b808b1d15d74` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:54.284017 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13372770-9eff-4f45-9708-7c7452469a68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f051cb0057c14a5891a0b808b1d15d74)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:54.286655 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13372770-9eff-4f45-9708-7c7452469a68 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:54.376632 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-13372770-9eff-4f45-9708-7c7452469a68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f051cb0057c14a5891a0b808b1d15d74', 'request_id': 'req-13372770-9eff-4f45-9708-7c7452469a68'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:54.389447 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-13372770-9eff-4f45-9708-7c7452469a68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f051cb0057c14a5891a0b808b1d15d74', 'request_id': 'req-13372770-9eff-4f45-9708-7c7452469a68'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:54.405656 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-13372770-9eff-4f45-9708-7c7452469a68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f051cb0057c14a5891a0b808b1d15d74', 'request_id': 'req-13372770-9eff-4f45-9708-7c7452469a68'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:54.423668 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-13372770-9eff-4f45-9708-7c7452469a68 admin admin] The token cache is being invalidate because project f051cb0057c14a5891a0b808b1d15d74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:27:54.424243 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-13372770-9eff-4f45-9708-7c7452469a68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:54.425259 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-13372770-9eff-4f45-9708-7c7452469a68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:54.430626 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1000/1998] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:27:54 2022] DELETE /identity/v3/projects/f051cb0057c14a5891a0b808b1d15d74 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:27:54.435389 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa6ed1-fb6f-4656-8c04-68d5b61e156a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:54.435676 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa6ed1-fb6f-4656-8c04-68d5b61e156a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:54.435950 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa6ed1-fb6f-4656-8c04-68d5b61e156a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:54.465548 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-17aa6ed1-fb6f-4656-8c04-68d5b61e156a None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:27:54.466957 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-17aa6ed1-fb6f-4656-8c04-68d5b61e156a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:54.571395 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 999/1999] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:27:54.576821 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e040ee4-76d9-421f-997f-d737a865903f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:54.577258 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e040ee4-76d9-421f-997f-d737a865903f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:54.577373 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e040ee4-76d9-421f-997f-d737a865903f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:54.611977 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1e040ee4-76d9-421f-997f-d737a865903f None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:27:54.613771 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1e040ee4-76d9-421f-997f-d737a865903f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:54.631388 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1001/2000] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:27:54.637310 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aa6ed1-fb6f-4656-8c04-68d5b61e156a None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:54.668250 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aa6ed1-fb6f-4656-8c04-68d5b61e156a None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:54.683195 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c671d0de-2791-4b7e-9765-58d50af00684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:54.685101 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671d0de-2791-4b7e-9765-58d50af00684 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:54.686834 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671d0de-2791-4b7e-9765-58d50af00684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:54.687789 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671d0de-2791-4b7e-9765-58d50af00684 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:54.689442 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c671d0de-2791-4b7e-9765-58d50af00684 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:27:54.690782 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c671d0de-2791-4b7e-9765-58d50af00684 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:54.695066 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c671d0de-2791-4b7e-9765-58d50af00684 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:54.730487 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1000/2001] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:27:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:27:54.736033 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e040ee4-76d9-421f-997f-d737a865903f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:54.740644 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:27:54.750609 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e040ee4-76d9-421f-997f-d737a865903f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:54.763786 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bc5d3f-2b8a-4ddc-8622-9544bdba8cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:54.764542 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc5d3f-2b8a-4ddc-8622-9544bdba8cb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:54.764948 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc5d3f-2b8a-4ddc-8622-9544bdba8cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:54.765323 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc5d3f-2b8a-4ddc-8622-9544bdba8cb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:54.768054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bc5d3f-2b8a-4ddc-8622-9544bdba8cb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:54.770099 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bc5d3f-2b8a-4ddc-8622-9544bdba8cb7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:54.830674 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1002/2002] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:27:54 2022] POST /identity/v3/projects => generated 383 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:27:54.832743 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c671d0de-2791-4b7e-9765-58d50af00684 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:54.845648 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c671d0de-2791-4b7e-9765-58d50af00684 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:54.894250 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:27:54.978120 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df9fe7-0597-46b8-82e2-73b2b887170f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:54.981079 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df9fe7-0597-46b8-82e2-73b2b887170f admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:54.981537 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df9fe7-0597-46b8-82e2-73b2b887170f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:54.982554 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df9fe7-0597-46b8-82e2-73b2b887170f admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:54.983947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56df9fe7-0597-46b8-82e2-73b2b887170f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:54.986117 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56df9fe7-0597-46b8-82e2-73b2b887170f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:55.049718 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1001/2003] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:27:54 2022] POST /identity/v3/users => generated 292 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:27:55.052189 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bc5d3f-2b8a-4ddc-8622-9544bdba8cb7 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:55.069023 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bc5d3f-2b8a-4ddc-8622-9544bdba8cb7 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:55.085485 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c96444-5ad5-46a3-bea0-8fcd86f094a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:55.086242 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c96444-5ad5-46a3-bea0-8fcd86f094a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:55.086841 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c96444-5ad5-46a3-bea0-8fcd86f094a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:55.086991 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c96444-5ad5-46a3-bea0-8fcd86f094a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:55.087760 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c96444-5ad5-46a3-bea0-8fcd86f094a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:55.089498 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c96444-5ad5-46a3-bea0-8fcd86f094a0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:55.111971 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df9fe7-0597-46b8-82e2-73b2b887170f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:55.122970 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1003/2004] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:27:55.125870 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df9fe7-0597-46b8-82e2-73b2b887170f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:55.131185 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7da09aa-4d57-41bb-87e9-3059800f9d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:55.132037 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da09aa-4d57-41bb-87e9-3059800f9d26 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:55.132391 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da09aa-4d57-41bb-87e9-3059800f9d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:55.133119 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da09aa-4d57-41bb-87e9-3059800f9d26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:55.134376 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7da09aa-4d57-41bb-87e9-3059800f9d26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:55.136442 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7da09aa-4d57-41bb-87e9-3059800f9d26 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:55.172429 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c96444-5ad5-46a3-bea0-8fcd86f094a0 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:55.174382 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1002/2005] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:27:55.231429 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c96444-5ad5-46a3-bea0-8fcd86f094a0 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:55.241365 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a369af0-54b0-401d-9abe-85d58bf624a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:55.242665 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a369af0-54b0-401d-9abe-85d58bf624a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:55.243106 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a369af0-54b0-401d-9abe-85d58bf624a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:55.243488 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a369af0-54b0-401d-9abe-85d58bf624a8 admin admin] PATH_INFO: `/v3/projects/af14c4d7e67b4dafa30a72fa045cae1b/users/7c6c659439924f3c8fbc7b225f4b795a/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:55.292702 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a369af0-54b0-401d-9abe-85d58bf624a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af14c4d7e67b4dafa30a72fa045cae1b, user_id=7c6c659439924f3c8fbc7b225f4b795a, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:55.296351 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a369af0-54b0-401d-9abe-85d58bf624a8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:55.409429 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1004/2006] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:27:55 2022] PUT /identity/v3/projects/af14c4d7e67b4dafa30a72fa045cae1b/users/7c6c659439924f3c8fbc7b225f4b795a/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:27:55.424253 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd655c-76db-4f09-81d9-c53f8831d180 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:55.431783 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd655c-76db-4f09-81d9-c53f8831d180 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:55.431783 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd655c-76db-4f09-81d9-c53f8831d180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:55.466260 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-15fd655c-76db-4f09-81d9-c53f8831d180 None None] MFA Rules not processed for user `7c6c659439924f3c8fbc7b225f4b795a`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:27:55.467862 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-15fd655c-76db-4f09-81d9-c53f8831d180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:55.563774 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1003/2007] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:27:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:27:55.567863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a369af0-54b0-401d-9abe-85d58bf624a8 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:55.677048 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a369af0-54b0-401d-9abe-85d58bf624a8 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:55.693548 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec2a2f1-6e0a-40f6-8392-d866b9f1255d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:55.694326 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec2a2f1-6e0a-40f6-8392-d866b9f1255d admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:55.694681 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec2a2f1-6e0a-40f6-8392-d866b9f1255d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:55.695220 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec2a2f1-6e0a-40f6-8392-d866b9f1255d admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:55.696443 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec2a2f1-6e0a-40f6-8392-d866b9f1255d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:55.702041 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec2a2f1-6e0a-40f6-8392-d866b9f1255d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:55.754669 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1005/2008] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:27:55 2022] POST /identity/v3/projects => generated 381 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:27:55.755961 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd655c-76db-4f09-81d9-c53f8831d180 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:55.793152 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd655c-76db-4f09-81d9-c53f8831d180 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:55.913193 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3042d109-5b7c-4351-a59b-adfa76661f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:55.914078 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3042d109-5b7c-4351-a59b-adfa76661f21 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:55.914365 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3042d109-5b7c-4351-a59b-adfa76661f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:55.914931 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3042d109-5b7c-4351-a59b-adfa76661f21 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:55.915744 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3042d109-5b7c-4351-a59b-adfa76661f21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:55.917968 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3042d109-5b7c-4351-a59b-adfa76661f21 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:55.964943 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1004/2009] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:27:55 2022] POST /identity/v3/users => generated 297 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:27:55.967128 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec2a2f1-6e0a-40f6-8392-d866b9f1255d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:55.986544 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec2a2f1-6e0a-40f6-8392-d866b9f1255d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:55.995904 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10992d8-37e9-4848-8176-47349be6a338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:55.996963 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10992d8-37e9-4848-8176-47349be6a338 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:55.997413 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10992d8-37e9-4848-8176-47349be6a338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:55.997696 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10992d8-37e9-4848-8176-47349be6a338 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:56.001554 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10992d8-37e9-4848-8176-47349be6a338 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:56.004924 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10992d8-37e9-4848-8176-47349be6a338 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:56.026562 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1006/2010] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 10:27:56.027813 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3042d109-5b7c-4351-a59b-adfa76661f21 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:56.048756 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3042d109-5b7c-4351-a59b-adfa76661f21 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:56.054534 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c16ef89-793b-47b0-b4e3-ec1ca8f6fbf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:56.055828 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c16ef89-793b-47b0-b4e3-ec1ca8f6fbf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:56.056142 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c16ef89-793b-47b0-b4e3-ec1ca8f6fbf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:56.056392 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c16ef89-793b-47b0-b4e3-ec1ca8f6fbf5 admin admin] PATH_INFO: `/v3/projects/7a9032c685ff439385628c36cb1ff5af/users/c9143400a7504c14a6acb5111dd009f7/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:56.085061 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c16ef89-793b-47b0-b4e3-ec1ca8f6fbf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a9032c685ff439385628c36cb1ff5af, user_id=c9143400a7504c14a6acb5111dd009f7, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:56.089100 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c16ef89-793b-47b0-b4e3-ec1ca8f6fbf5 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:56.113521 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1005/2011] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:27:56 2022] PUT /identity/v3/projects/7a9032c685ff439385628c36cb1ff5af/users/c9143400a7504c14a6acb5111dd009f7/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:27:56.117876 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10992d8-37e9-4848-8176-47349be6a338 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:56.130990 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10992d8-37e9-4848-8176-47349be6a338 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:56.224296 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd54a452-135b-41b4-95c9-871bda4db826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:56.225671 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd54a452-135b-41b4-95c9-871bda4db826 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:56.226053 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd54a452-135b-41b4-95c9-871bda4db826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:56.226417 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd54a452-135b-41b4-95c9-871bda4db826 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:56.229029 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd54a452-135b-41b4-95c9-871bda4db826 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:56.231797 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd54a452-135b-41b4-95c9-871bda4db826 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:56.269442 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1007/2012] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:27:56.270867 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c16ef89-793b-47b0-b4e3-ec1ca8f6fbf5 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:56.276788 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:27:56.292313 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c16ef89-793b-47b0-b4e3-ec1ca8f6fbf5 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:56.301335 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f925f42-3106-40e0-b5e3-3d47cbfe4de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:56.302346 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f925f42-3106-40e0-b5e3-3d47cbfe4de7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:56.302849 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f925f42-3106-40e0-b5e3-3d47cbfe4de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:56.303262 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f925f42-3106-40e0-b5e3-3d47cbfe4de7 admin admin] PATH_INFO: `/v3/projects/7a9032c685ff439385628c36cb1ff5af/users/c9143400a7504c14a6acb5111dd009f7/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:56.305778 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f925f42-3106-40e0-b5e3-3d47cbfe4de7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a9032c685ff439385628c36cb1ff5af, user_id=c9143400a7504c14a6acb5111dd009f7, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:56.308984 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f925f42-3106-40e0-b5e3-3d47cbfe4de7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:56.346100 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1006/2013] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:27:56 2022] PUT /identity/v3/projects/7a9032c685ff439385628c36cb1ff5af/users/c9143400a7504c14a6acb5111dd009f7/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:27:56.349420 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbb73b-bc8d-4979-a97a-e403aef9326a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:56.349808 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbb73b-bc8d-4979-a97a-e403aef9326a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:56.350181 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbb73b-bc8d-4979-a97a-e403aef9326a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:56.398366 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e2fbb73b-bc8d-4979-a97a-e403aef9326a None None] MFA Rules not processed for user `c9143400a7504c14a6acb5111dd009f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:27:56.399884 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e2fbb73b-bc8d-4979-a97a-e403aef9326a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:56.404973 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:27:56.498215 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1008/2014] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:27:56 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:27:56.510525 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f925f42-3106-40e0-b5e3-3d47cbfe4de7 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:56.512472 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3f925f42-3106-40e0-b5e3-3d47cbfe4de7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:56.580115 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f224a-2596-4d0e-84db-db334b551080 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:56.580598 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f224a-2596-4d0e-84db-db334b551080 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:56.580995 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f224a-2596-4d0e-84db-db334b551080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:56.599240 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f925f42-3106-40e0-b5e3-3d47cbfe4de7 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:56.601946 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b22f224a-2596-4d0e-84db-db334b551080 None None] MFA Rules not processed for user `9697af06b1104425aed6692ba4f29238`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:27:56.603186 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d3854-0fda-4041-b301-da2283722f52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:56.603316 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b22f224a-2596-4d0e-84db-db334b551080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:56.604509 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d3854-0fda-4041-b301-da2283722f52 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:56.604800 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d3854-0fda-4041-b301-da2283722f52 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:56.605141 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d3854-0fda-4041-b301-da2283722f52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:56.616263 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d3854-0fda-4041-b301-da2283722f52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:56.618222 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d3854-0fda-4041-b301-da2283722f52 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:56.629422 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1007/2015] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:27:56 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:27:56.710976 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1009/2016] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:27:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:27:56.769084 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d3854-0fda-4041-b301-da2283722f52 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:56.785644 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d3854-0fda-4041-b301-da2283722f52 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:56.790262 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92200f59-9db9-4158-b3c1-a5c0971520d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:56.791814 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92200f59-9db9-4158-b3c1-a5c0971520d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:56.796825 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92200f59-9db9-4158-b3c1-a5c0971520d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:56.797327 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92200f59-9db9-4158-b3c1-a5c0971520d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:56.859444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92200f59-9db9-4158-b3c1-a5c0971520d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:56.860944 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92200f59-9db9-4158-b3c1-a5c0971520d9 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:56.871065 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1008/2017] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:27:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:27:57.239352 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f224a-2596-4d0e-84db-db334b551080 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:57.340839 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f224a-2596-4d0e-84db-db334b551080 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:57.346535 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce69191-290c-424c-85e5-615467ba2841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:57.347873 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce69191-290c-424c-85e5-615467ba2841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:57.348229 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce69191-290c-424c-85e5-615467ba2841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:57.348660 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce69191-290c-424c-85e5-615467ba2841 admin admin] PATH_INFO: `/v3/users/7c6c659439924f3c8fbc7b225f4b795a` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:57.349943 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce69191-290c-424c-85e5-615467ba2841 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c6c659439924f3c8fbc7b225f4b795a)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:57.351945 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce69191-290c-424c-85e5-615467ba2841 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:57.453820 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9ce69191-290c-424c-85e5-615467ba2841 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c6c659439924f3c8fbc7b225f4b795a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:57.454843 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9ce69191-290c-424c-85e5-615467ba2841 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c6c659439924f3c8fbc7b225f4b795a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:57.455148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9ce69191-290c-424c-85e5-615467ba2841 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c6c659439924f3c8fbc7b225f4b795a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:57.459281 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:27:57.473518 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9ce69191-290c-424c-85e5-615467ba2841 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c6c659439924f3c8fbc7b225f4b795a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:57.483377 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9ce69191-290c-424c-85e5-615467ba2841 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c6c659439924f3c8fbc7b225f4b795a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:57.525700 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1010/2018] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:27:57 2022] DELETE /identity/v3/users/7c6c659439924f3c8fbc7b225f4b795a => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:27:57.530228 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb258a3-a4a4-4c55-8591-5579f143dad6 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:57.531016 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb258a3-a4a4-4c55-8591-5579f143dad6 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:57.531603 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb258a3-a4a4-4c55-8591-5579f143dad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:57.564380 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-7eb258a3-a4a4-4c55-8591-5579f143dad6 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:27:57.565940 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7eb258a3-a4a4-4c55-8591-5579f143dad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:57.664905 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1009/2019] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:27:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:27:57.672518 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce69191-290c-424c-85e5-615467ba2841 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:57.674234 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9ce69191-290c-424c-85e5-615467ba2841 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:57.782507 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce69191-290c-424c-85e5-615467ba2841 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:57.789475 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6e32b3-d05d-47e8-9701-ac178a67c0a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:57.790138 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6e32b3-d05d-47e8-9701-ac178a67c0a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:57.790414 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6e32b3-d05d-47e8-9701-ac178a67c0a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:57.790689 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6e32b3-d05d-47e8-9701-ac178a67c0a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:57.801672 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6e32b3-d05d-47e8-9701-ac178a67c0a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:57.803397 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6e32b3-d05d-47e8-9701-ac178a67c0a0 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:57.830303 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1011/2020] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:27:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:27:58.061061 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb258a3-a4a4-4c55-8591-5579f143dad6 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:58.062580 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7eb258a3-a4a4-4c55-8591-5579f143dad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:58.079341 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb258a3-a4a4-4c55-8591-5579f143dad6 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:58.087681 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da8f863-f138-4ba7-942e-44e58b732116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:58.088405 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8f863-f138-4ba7-942e-44e58b732116 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:58.089115 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8f863-f138-4ba7-942e-44e58b732116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:58.089419 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8f863-f138-4ba7-942e-44e58b732116 admin admin] PATH_INFO: `/v3/users/9ace48889c484e4fb5fed0f8f80c3288` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:58.091287 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da8f863-f138-4ba7-942e-44e58b732116 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ace48889c484e4fb5fed0f8f80c3288)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:58.093434 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da8f863-f138-4ba7-942e-44e58b732116 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:58.132230 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:27:58.160788 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8da8f863-f138-4ba7-942e-44e58b732116 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ace48889c484e4fb5fed0f8f80c3288'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:58.172990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8da8f863-f138-4ba7-942e-44e58b732116 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ace48889c484e4fb5fed0f8f80c3288'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:58.173764 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8da8f863-f138-4ba7-942e-44e58b732116 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ace48889c484e4fb5fed0f8f80c3288'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:58.192773 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8da8f863-f138-4ba7-942e-44e58b732116 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ace48889c484e4fb5fed0f8f80c3288'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:58.193492 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8da8f863-f138-4ba7-942e-44e58b732116 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ace48889c484e4fb5fed0f8f80c3288'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:58.209267 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6e32b3-d05d-47e8-9701-ac178a67c0a0 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:58.213546 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1010/2021] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:27:58 2022] DELETE /identity/v3/users/9ace48889c484e4fb5fed0f8f80c3288 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:27:58.213724 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7c6e32b3-d05d-47e8-9701-ac178a67c0a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:58.238996 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6e32b3-d05d-47e8-9701-ac178a67c0a0 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:58.317605 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90b09c-3c58-4def-99e3-d97415b2237c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:58.318482 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90b09c-3c58-4def-99e3-d97415b2237c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:58.319563 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90b09c-3c58-4def-99e3-d97415b2237c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:58.320001 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90b09c-3c58-4def-99e3-d97415b2237c admin admin] PATH_INFO: `/v3/projects/af14c4d7e67b4dafa30a72fa045cae1b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:58.321487 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df90b09c-3c58-4def-99e3-d97415b2237c admin admin] RBAC: Authorizing `identity:delete_project(project_id=af14c4d7e67b4dafa30a72fa045cae1b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:58.323666 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df90b09c-3c58-4def-99e3-d97415b2237c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:58.373551 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-df90b09c-3c58-4def-99e3-d97415b2237c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af14c4d7e67b4dafa30a72fa045cae1b', 'request_id': 'req-df90b09c-3c58-4def-99e3-d97415b2237c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:58.384127 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-df90b09c-3c58-4def-99e3-d97415b2237c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af14c4d7e67b4dafa30a72fa045cae1b', 'request_id': 'req-df90b09c-3c58-4def-99e3-d97415b2237c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:58.390202 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-df90b09c-3c58-4def-99e3-d97415b2237c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af14c4d7e67b4dafa30a72fa045cae1b', 'request_id': 'req-df90b09c-3c58-4def-99e3-d97415b2237c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:58.397783 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:27:58.405699 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-df90b09c-3c58-4def-99e3-d97415b2237c admin admin] The token cache is being invalidate because project af14c4d7e67b4dafa30a72fa045cae1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:27:58.406150 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da8f863-f138-4ba7-942e-44e58b732116 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:58.406295 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-df90b09c-3c58-4def-99e3-d97415b2237c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:58.407135 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-df90b09c-3c58-4def-99e3-d97415b2237c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:58.407808 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8da8f863-f138-4ba7-942e-44e58b732116 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:58.409432 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1012/2022] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:27:58 2022] DELETE /identity/v3/projects/af14c4d7e67b4dafa30a72fa045cae1b => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:27:58.413525 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90b09c-3c58-4def-99e3-d97415b2237c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:58.420533 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-df90b09c-3c58-4def-99e3-d97415b2237c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:58.427254 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da8f863-f138-4ba7-942e-44e58b732116 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:58.487823 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8525f23-719d-42bd-b84e-fea481ea37c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:58.488546 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8525f23-719d-42bd-b84e-fea481ea37c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:58.488876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8525f23-719d-42bd-b84e-fea481ea37c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:58.489591 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8525f23-719d-42bd-b84e-fea481ea37c3 admin admin] PATH_INFO: `/v3/projects/5a7a56c0fd0c43d4803634f30479187a` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:58.489735 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90b09c-3c58-4def-99e3-d97415b2237c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:58.490589 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8525f23-719d-42bd-b84e-fea481ea37c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a7a56c0fd0c43d4803634f30479187a)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:58.493444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8525f23-719d-42bd-b84e-fea481ea37c3 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:58.495273 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26e4b6b-5c4c-4fb0-aa0c-fd88745bdf6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:58.496012 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e4b6b-5c4c-4fb0-aa0c-fd88745bdf6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:58.496297 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e4b6b-5c4c-4fb0-aa0c-fd88745bdf6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:58.496573 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e4b6b-5c4c-4fb0-aa0c-fd88745bdf6a admin admin] PATH_INFO: `/v3/users/c9143400a7504c14a6acb5111dd009f7` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:58.497827 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26e4b6b-5c4c-4fb0-aa0c-fd88745bdf6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9143400a7504c14a6acb5111dd009f7)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:58.499893 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26e4b6b-5c4c-4fb0-aa0c-fd88745bdf6a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:58.544342 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-b8525f23-719d-42bd-b84e-fea481ea37c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a7a56c0fd0c43d4803634f30479187a', 'request_id': 'req-b8525f23-719d-42bd-b84e-fea481ea37c3'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:58.551080 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-b8525f23-719d-42bd-b84e-fea481ea37c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a7a56c0fd0c43d4803634f30479187a', 'request_id': 'req-b8525f23-719d-42bd-b84e-fea481ea37c3'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:58.552146 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c26e4b6b-5c4c-4fb0-aa0c-fd88745bdf6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9143400a7504c14a6acb5111dd009f7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:58.552819 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c26e4b6b-5c4c-4fb0-aa0c-fd88745bdf6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9143400a7504c14a6acb5111dd009f7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:58.553487 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c26e4b6b-5c4c-4fb0-aa0c-fd88745bdf6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9143400a7504c14a6acb5111dd009f7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:58.562232 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-b8525f23-719d-42bd-b84e-fea481ea37c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a7a56c0fd0c43d4803634f30479187a', 'request_id': 'req-b8525f23-719d-42bd-b84e-fea481ea37c3'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:58.565796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c26e4b6b-5c4c-4fb0-aa0c-fd88745bdf6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9143400a7504c14a6acb5111dd009f7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:58.573754 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c26e4b6b-5c4c-4fb0-aa0c-fd88745bdf6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9143400a7504c14a6acb5111dd009f7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:58.574410 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-b8525f23-719d-42bd-b84e-fea481ea37c3 admin admin] The token cache is being invalidate because project 5a7a56c0fd0c43d4803634f30479187a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:27:58.574702 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-b8525f23-719d-42bd-b84e-fea481ea37c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:58.575498 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-b8525f23-719d-42bd-b84e-fea481ea37c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:58.577347 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1011/2023] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:27:58 2022] DELETE /identity/v3/projects/5a7a56c0fd0c43d4803634f30479187a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:27:58.583648 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bf27fa-1608-4b0f-b6da-665fa35c43b7 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:58.583937 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bf27fa-1608-4b0f-b6da-665fa35c43b7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:58.584217 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bf27fa-1608-4b0f-b6da-665fa35c43b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:58.596082 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1013/2024] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:27:58 2022] DELETE /identity/v3/users/c9143400a7504c14a6acb5111dd009f7 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:27:58.604338 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a4bf27fa-1608-4b0f-b6da-665fa35c43b7 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:27:58.605561 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a4bf27fa-1608-4b0f-b6da-665fa35c43b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:58.635081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:27:58.670419 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1012/2025] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:27:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:27:58.679119 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca6aed-64f2-429d-b7ad-0e1a167a676b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:58.679501 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca6aed-64f2-429d-b7ad-0e1a167a676b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:58.680451 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca6aed-64f2-429d-b7ad-0e1a167a676b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:58.702318 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-86ca6aed-64f2-429d-b7ad-0e1a167a676b None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:27:58.703578 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-86ca6aed-64f2-429d-b7ad-0e1a167a676b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:58.711845 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1014/2026] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:27:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:27:58.716683 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bf27fa-1608-4b0f-b6da-665fa35c43b7 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:58.727744 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bf27fa-1608-4b0f-b6da-665fa35c43b7 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:58.733221 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee0f911-7586-4c90-b36d-0614655a5290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:58.734367 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0f911-7586-4c90-b36d-0614655a5290 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:58.734832 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0f911-7586-4c90-b36d-0614655a5290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:58.736346 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0f911-7586-4c90-b36d-0614655a5290 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:58.736346 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee0f911-7586-4c90-b36d-0614655a5290 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:27:58.736346 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee0f911-7586-4c90-b36d-0614655a5290 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:58.737551 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee0f911-7586-4c90-b36d-0614655a5290 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:58.755609 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1013/2027] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:27:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:27:58.760471 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ca6aed-64f2-429d-b7ad-0e1a167a676b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:58.770400 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ca6aed-64f2-429d-b7ad-0e1a167a676b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:58.775714 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1859d5-f471-4847-b8de-491fe6ca601c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:58.776353 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1859d5-f471-4847-b8de-491fe6ca601c admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:58.776619 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1859d5-f471-4847-b8de-491fe6ca601c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:58.776909 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1859d5-f471-4847-b8de-491fe6ca601c admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:58.777579 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1859d5-f471-4847-b8de-491fe6ca601c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:58.779540 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1859d5-f471-4847-b8de-491fe6ca601c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:58.815007 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1015/2028] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:27:58 2022] POST /identity/v3/projects => generated 365 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:27:58.815913 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee0f911-7586-4c90-b36d-0614655a5290 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:58.828244 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee0f911-7586-4c90-b36d-0614655a5290 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:58.881517 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1859d5-f471-4847-b8de-491fe6ca601c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:58.883993 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6b1859d5-f471-4847-b8de-491fe6ca601c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:58.938048 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb78698-a696-40d2-9e10-91dfdb2d6a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:58.938755 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb78698-a696-40d2-9e10-91dfdb2d6a04 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:58.939161 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb78698-a696-40d2-9e10-91dfdb2d6a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:58.939565 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb78698-a696-40d2-9e10-91dfdb2d6a04 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:58.940413 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb78698-a696-40d2-9e10-91dfdb2d6a04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:58.942417 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb78698-a696-40d2-9e10-91dfdb2d6a04 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:58.976466 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1014/2029] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:27:58 2022] POST /identity/v3/users => generated 283 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:27:58.980830 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb78698-a696-40d2-9e10-91dfdb2d6a04 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:58.993416 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:27:59.002352 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb78698-a696-40d2-9e10-91dfdb2d6a04 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:59.009117 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c353e2fa-557c-48ed-b7ef-bb5935376c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:59.009937 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353e2fa-557c-48ed-b7ef-bb5935376c1e admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:59.010326 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353e2fa-557c-48ed-b7ef-bb5935376c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:59.010660 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353e2fa-557c-48ed-b7ef-bb5935376c1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:59.011516 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c353e2fa-557c-48ed-b7ef-bb5935376c1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:59.013277 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c353e2fa-557c-48ed-b7ef-bb5935376c1e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:59.029114 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1859d5-f471-4847-b8de-491fe6ca601c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:59.032208 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1015/2030] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:27:59.033142 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c353e2fa-557c-48ed-b7ef-bb5935376c1e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:59.040061 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4d8270-c889-47c1-804b-5e5b30872442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:59.041056 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d8270-c889-47c1-804b-5e5b30872442 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:59.042291 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d8270-c889-47c1-804b-5e5b30872442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:59.042640 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d8270-c889-47c1-804b-5e5b30872442 admin admin] PATH_INFO: `/v3/projects/7a9032c685ff439385628c36cb1ff5af` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:59.045669 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4d8270-c889-47c1-804b-5e5b30872442 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a9032c685ff439385628c36cb1ff5af)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:59.045848 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c353e2fa-557c-48ed-b7ef-bb5935376c1e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:59.047702 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4d8270-c889-47c1-804b-5e5b30872442 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:59.053124 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9affb8-5b60-4be4-8161-b188f5e203ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:59.054032 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9affb8-5b60-4be4-8161-b188f5e203ee admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:59.054259 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9affb8-5b60-4be4-8161-b188f5e203ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:59.054750 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9affb8-5b60-4be4-8161-b188f5e203ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:59.055628 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9affb8-5b60-4be4-8161-b188f5e203ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:59.057396 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9affb8-5b60-4be4-8161-b188f5e203ee admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:59.075454 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1016/2031] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:27:59 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:27:59.080211 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9affb8-5b60-4be4-8161-b188f5e203ee admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:59.094946 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9affb8-5b60-4be4-8161-b188f5e203ee admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:59.102215 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c859700-e23c-43bc-8561-4c53b19c8af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:59.103101 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c859700-e23c-43bc-8561-4c53b19c8af9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:59.103411 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c859700-e23c-43bc-8561-4c53b19c8af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:59.103780 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c859700-e23c-43bc-8561-4c53b19c8af9 admin admin] PATH_INFO: `/v3/projects/516e7abb460f49d3ac19610315c31e92/users/1650fd8e00904b8b8925f84e800667db/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:59.122788 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1f4d8270-c889-47c1-804b-5e5b30872442 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a9032c685ff439385628c36cb1ff5af', 'request_id': 'req-1f4d8270-c889-47c1-804b-5e5b30872442'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:59.124296 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c859700-e23c-43bc-8561-4c53b19c8af9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=516e7abb460f49d3ac19610315c31e92, user_id=1650fd8e00904b8b8925f84e800667db, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:59.127502 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c859700-e23c-43bc-8561-4c53b19c8af9 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:59.132541 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1f4d8270-c889-47c1-804b-5e5b30872442 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a9032c685ff439385628c36cb1ff5af', 'request_id': 'req-1f4d8270-c889-47c1-804b-5e5b30872442'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:59.139962 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1f4d8270-c889-47c1-804b-5e5b30872442 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a9032c685ff439385628c36cb1ff5af', 'request_id': 'req-1f4d8270-c889-47c1-804b-5e5b30872442'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:59.143599 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1017/2032] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:27:59 2022] PUT /identity/v3/projects/516e7abb460f49d3ac19610315c31e92/users/1650fd8e00904b8b8925f84e800667db/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:27:59.154003 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a8881-d2bf-4486-ad25-df2ac902eaf5 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:59.154291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a8881-d2bf-4486-ad25-df2ac902eaf5 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:59.154572 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a8881-d2bf-4486-ad25-df2ac902eaf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:59.160697 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1f4d8270-c889-47c1-804b-5e5b30872442 admin admin] The token cache is being invalidate because project 7a9032c685ff439385628c36cb1ff5af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:27:59.161085 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1f4d8270-c889-47c1-804b-5e5b30872442 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:59.161832 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1f4d8270-c889-47c1-804b-5e5b30872442 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:27:59.164181 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1016/2033] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:27:58 2022] DELETE /identity/v3/projects/7a9032c685ff439385628c36cb1ff5af => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:27:59.168316 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472814d0-c864-4588-8abd-c57b4944946c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:59.168670 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472814d0-c864-4588-8abd-c57b4944946c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:59.169010 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472814d0-c864-4588-8abd-c57b4944946c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:59.183540 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-f92a8881-d2bf-4486-ad25-df2ac902eaf5 None None] MFA Rules not processed for user `1650fd8e00904b8b8925f84e800667db`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:27:59.184762 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f92a8881-d2bf-4486-ad25-df2ac902eaf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:59.190040 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-472814d0-c864-4588-8abd-c57b4944946c None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:27:59.191217 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-472814d0-c864-4588-8abd-c57b4944946c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:59.240746 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:27:59.250171 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1017/2034] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:27:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:27:59.253114 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1018/2035] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:27:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:27:59.257129 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a8881-d2bf-4486-ad25-df2ac902eaf5 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:59.257821 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b25b3-0897-4c08-b321-d60437364b9c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:59.258246 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b25b3-0897-4c08-b321-d60437364b9c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:59.258389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b25b3-0897-4c08-b321-d60437364b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:59.258507 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f92a8881-d2bf-4486-ad25-df2ac902eaf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:59.271988 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a8881-d2bf-4486-ad25-df2ac902eaf5 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:59.276974 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec58f7e-d76f-43c8-af77-6440cc0249ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:59.277702 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec58f7e-d76f-43c8-af77-6440cc0249ee admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:59.278004 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec58f7e-d76f-43c8-af77-6440cc0249ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:59.278101 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ed4b25b3-0897-4c08-b321-d60437364b9c None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:27:59.278263 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec58f7e-d76f-43c8-af77-6440cc0249ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:59.279055 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec58f7e-d76f-43c8-af77-6440cc0249ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:59.279186 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ed4b25b3-0897-4c08-b321-d60437364b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:59.281041 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec58f7e-d76f-43c8-af77-6440cc0249ee admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:59.287292 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1018/2036] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:27:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:27:59.291718 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4b25b3-0897-4c08-b321-d60437364b9c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:59.301351 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4b25b3-0897-4c08-b321-d60437364b9c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:59.306562 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e93d97-5116-4921-b8c4-7de0c9e65e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:59.307223 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e93d97-5116-4921-b8c4-7de0c9e65e68 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:59.307505 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e93d97-5116-4921-b8c4-7de0c9e65e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:59.307832 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e93d97-5116-4921-b8c4-7de0c9e65e68 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:59.308472 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e93d97-5116-4921-b8c4-7de0c9e65e68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:27:59.308778 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e93d97-5116-4921-b8c4-7de0c9e65e68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:59.313167 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1019/2037] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:27:59 2022] POST /identity/v3/projects => generated 367 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:27:59.313319 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e93d97-5116-4921-b8c4-7de0c9e65e68 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:59.314242 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec58f7e-d76f-43c8-af77-6440cc0249ee admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:59.326440 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1019/2038] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:27:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:27:59.327418 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e93d97-5116-4921-b8c4-7de0c9e65e68 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:59.337418 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e93d97-5116-4921-b8c4-7de0c9e65e68 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:59.385957 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec58f7e-d76f-43c8-af77-6440cc0249ee admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:59.394471 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc1e7c1-d30e-4798-947c-cd6a2fc03eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:59.395204 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc1e7c1-d30e-4798-947c-cd6a2fc03eca admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:59.395659 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc1e7c1-d30e-4798-947c-cd6a2fc03eca admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:59.395958 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc1e7c1-d30e-4798-947c-cd6a2fc03eca admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:59.396799 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc1e7c1-d30e-4798-947c-cd6a2fc03eca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:59.399220 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc1e7c1-d30e-4798-947c-cd6a2fc03eca admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:59.401896 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d86a748-259c-469c-ba0d-890902e13ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:59.402552 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d86a748-259c-469c-ba0d-890902e13ad3 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:59.402836 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d86a748-259c-469c-ba0d-890902e13ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:59.403146 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d86a748-259c-469c-ba0d-890902e13ad3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:59.403818 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d86a748-259c-469c-ba0d-890902e13ad3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:59.405710 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d86a748-259c-469c-ba0d-890902e13ad3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:59.411432 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:27:59.429248 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1020/2039] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:27:59 2022] POST /identity/v3/users => generated 290 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:27:59.430634 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc1e7c1-d30e-4798-947c-cd6a2fc03eca admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:59.442801 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc1e7c1-d30e-4798-947c-cd6a2fc03eca admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:59.445479 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1020/2040] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:27:59 2022] POST /identity/v3/projects => generated 369 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:27:59.450174 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d86a748-259c-469c-ba0d-890902e13ad3 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:59.450368 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd23d91-b2d4-4aea-8cda-f479bbb769af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:59.451157 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd23d91-b2d4-4aea-8cda-f479bbb769af admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:59.451434 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd23d91-b2d4-4aea-8cda-f479bbb769af admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:59.451741 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd23d91-b2d4-4aea-8cda-f479bbb769af admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:59.453160 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd23d91-b2d4-4aea-8cda-f479bbb769af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:59.455185 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd23d91-b2d4-4aea-8cda-f479bbb769af admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:59.468672 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1021/2041] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:27:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 10:27:59.473165 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd23d91-b2d4-4aea-8cda-f479bbb769af admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:59.484264 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd23d91-b2d4-4aea-8cda-f479bbb769af admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:59.517469 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d86a748-259c-469c-ba0d-890902e13ad3 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:59.522102 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3557b938-d80c-47d0-8db6-782f412a29bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:59.523133 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557b938-d80c-47d0-8db6-782f412a29bd admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:59.523133 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557b938-d80c-47d0-8db6-782f412a29bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:59.523380 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557b938-d80c-47d0-8db6-782f412a29bd admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:59.524060 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3557b938-d80c-47d0-8db6-782f412a29bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:59.526058 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3557b938-d80c-47d0-8db6-782f412a29bd admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:59.542236 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12473f7f-1f6a-4dec-bf33-fa5565fd461a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:59.542940 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12473f7f-1f6a-4dec-bf33-fa5565fd461a admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:59.543230 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12473f7f-1f6a-4dec-bf33-fa5565fd461a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:59.543504 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12473f7f-1f6a-4dec-bf33-fa5565fd461a admin admin] PATH_INFO: `/v3/projects/3f25425a02634e8fa3ff7658d226cfbb/users/32e450631cf34293abd406edbe3abb8a/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:59.548886 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1021/2042] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:27:59 2022] POST /identity/v3/users => generated 285 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:27:59.553109 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3557b938-d80c-47d0-8db6-782f412a29bd admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:59.558336 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12473f7f-1f6a-4dec-bf33-fa5565fd461a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f25425a02634e8fa3ff7658d226cfbb, user_id=32e450631cf34293abd406edbe3abb8a, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:59.561031 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12473f7f-1f6a-4dec-bf33-fa5565fd461a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:59.562755 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3557b938-d80c-47d0-8db6-782f412a29bd admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:59.568048 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fab9ec-2349-484a-a76c-6663132564ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:59.568727 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fab9ec-2349-484a-a76c-6663132564ae admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:59.569012 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fab9ec-2349-484a-a76c-6663132564ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:59.569285 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fab9ec-2349-484a-a76c-6663132564ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:59.570079 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fab9ec-2349-484a-a76c-6663132564ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:59.572048 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fab9ec-2349-484a-a76c-6663132564ae admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:59.575279 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1022/2043] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:27:59 2022] PUT /identity/v3/projects/3f25425a02634e8fa3ff7658d226cfbb/users/32e450631cf34293abd406edbe3abb8a/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:27:59.579251 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12473f7f-1f6a-4dec-bf33-fa5565fd461a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:59.585749 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1022/2044] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:27:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:27:59.586815 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fab9ec-2349-484a-a76c-6663132564ae admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:59.596601 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fab9ec-2349-484a-a76c-6663132564ae admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:59.614947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:27:59.644091 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12473f7f-1f6a-4dec-bf33-fa5565fd461a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:59.648868 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46021a95-7b3c-41be-8227-54bcdbd48e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:59.649494 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46021a95-7b3c-41be-8227-54bcdbd48e17 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:59.649771 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46021a95-7b3c-41be-8227-54bcdbd48e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:59.650040 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46021a95-7b3c-41be-8227-54bcdbd48e17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:59.650732 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46021a95-7b3c-41be-8227-54bcdbd48e17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:59.652352 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46021a95-7b3c-41be-8227-54bcdbd48e17 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:59.656070 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995151d4-a043-4191-a142-0aeaafb921f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:59.656771 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995151d4-a043-4191-a142-0aeaafb921f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:59.657058 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995151d4-a043-4191-a142-0aeaafb921f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:59.657331 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995151d4-a043-4191-a142-0aeaafb921f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:59.658454 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995151d4-a043-4191-a142-0aeaafb921f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:59.660567 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995151d4-a043-4191-a142-0aeaafb921f6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:59.662576 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1023/2045] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:27:59 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 10:27:59.667106 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46021a95-7b3c-41be-8227-54bcdbd48e17 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:59.671152 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1023/2046] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:27:59 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 10:27:59.675343 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995151d4-a043-4191-a142-0aeaafb921f6 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:59.676713 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46021a95-7b3c-41be-8227-54bcdbd48e17 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:59.682031 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d925c9-6cf1-4a30-ae98-85ac7e704115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:59.682703 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d925c9-6cf1-4a30-ae98-85ac7e704115 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:59.683014 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d925c9-6cf1-4a30-ae98-85ac7e704115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:59.683288 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d925c9-6cf1-4a30-ae98-85ac7e704115 admin admin] PATH_INFO: `/v3/projects/3f25425a02634e8fa3ff7658d226cfbb/users/32e450631cf34293abd406edbe3abb8a/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:59.685313 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d925c9-6cf1-4a30-ae98-85ac7e704115 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f25425a02634e8fa3ff7658d226cfbb, user_id=32e450631cf34293abd406edbe3abb8a, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:59.685504 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995151d4-a043-4191-a142-0aeaafb921f6 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:59.687962 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d925c9-6cf1-4a30-ae98-85ac7e704115 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:59.690344 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7920eb4-e4eb-4016-993a-449f041b1aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:59.691060 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7920eb4-e4eb-4016-993a-449f041b1aa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:59.691345 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7920eb4-e4eb-4016-993a-449f041b1aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:59.691604 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7920eb4-e4eb-4016-993a-449f041b1aa7 admin admin] PATH_INFO: `/v3/projects/a4a8139ee97548e89fcb7b38c5bd99fe/users/7723cdbd378743378b1fccc2d15bc53b/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:59.698651 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1024/2047] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:27:59 2022] PUT /identity/v3/projects/3f25425a02634e8fa3ff7658d226cfbb/users/32e450631cf34293abd406edbe3abb8a/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:27:59.707620 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7920eb4-e4eb-4016-993a-449f041b1aa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4a8139ee97548e89fcb7b38c5bd99fe, user_id=7723cdbd378743378b1fccc2d15bc53b, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:59.710047 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756917d2-1163-4773-83cd-93fe8bd16ae6 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:59.710336 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756917d2-1163-4773-83cd-93fe8bd16ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:59.710677 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756917d2-1163-4773-83cd-93fe8bd16ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:59.710814 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7920eb4-e4eb-4016-993a-449f041b1aa7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:59.726170 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1024/2048] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:27:59 2022] PUT /identity/v3/projects/a4a8139ee97548e89fcb7b38c5bd99fe/users/7723cdbd378743378b1fccc2d15bc53b/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:27:59.731269 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24eb97f-5519-4855-9ed0-cb7343a51d68 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:59.731675 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24eb97f-5519-4855-9ed0-cb7343a51d68 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:59.731961 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24eb97f-5519-4855-9ed0-cb7343a51d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:59.740548 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-756917d2-1163-4773-83cd-93fe8bd16ae6 None None] MFA Rules not processed for user `32e450631cf34293abd406edbe3abb8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:27:59.741872 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-756917d2-1163-4773-83cd-93fe8bd16ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:59.750061 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a24eb97f-5519-4855-9ed0-cb7343a51d68 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:27:59.751274 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a24eb97f-5519-4855-9ed0-cb7343a51d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:59.783829 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:27:59.811835 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1025/2049] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:27:59 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:27:59.817576 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1025/2050] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:27:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:27:59.823751 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756917d2-1163-4773-83cd-93fe8bd16ae6 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:59.825182 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-756917d2-1163-4773-83cd-93fe8bd16ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:59.829413 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24eb97f-5519-4855-9ed0-cb7343a51d68 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:27:59.831413 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a24eb97f-5519-4855-9ed0-cb7343a51d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:27:59.889749 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756917d2-1163-4773-83cd-93fe8bd16ae6 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:59.893423 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f5726e-e3d6-4bf5-ad4d-6c54f645c788 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:59.894046 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f5726e-e3d6-4bf5-ad4d-6c54f645c788 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:59.894321 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f5726e-e3d6-4bf5-ad4d-6c54f645c788 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:59.894590 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f5726e-e3d6-4bf5-ad4d-6c54f645c788 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:59.896928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24eb97f-5519-4855-9ed0-cb7343a51d68 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:27:59.901227 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42b98f3-abb8-49bb-905d-938819038f88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:27:59.901888 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42b98f3-abb8-49bb-905d-938819038f88 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:27:59.902170 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42b98f3-abb8-49bb-905d-938819038f88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:27:59.902443 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42b98f3-abb8-49bb-905d-938819038f88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:27:59.903901 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f5726e-e3d6-4bf5-ad4d-6c54f645c788 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:59.905308 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f5726e-e3d6-4bf5-ad4d-6c54f645c788 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:59.913884 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42b98f3-abb8-49bb-905d-938819038f88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:27:59.915796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42b98f3-abb8-49bb-905d-938819038f88 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:27:59.917504 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1026/2051] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:27:59 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:27:59.928296 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1026/2052] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:27:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:28:00.019758 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f5726e-e3d6-4bf5-ad4d-6c54f645c788 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:00.031698 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f5726e-e3d6-4bf5-ad4d-6c54f645c788 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:00.042770 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76060c57-702e-4459-9c41-7f968e5088aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:00.043453 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76060c57-702e-4459-9c41-7f968e5088aa service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:00.043727 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76060c57-702e-4459-9c41-7f968e5088aa service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:00.043992 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76060c57-702e-4459-9c41-7f968e5088aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:00.146774 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:28:00.174398 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76060c57-702e-4459-9c41-7f968e5088aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:00.176227 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76060c57-702e-4459-9c41-7f968e5088aa service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:00.188623 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1027/2053] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:28:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:28:00.462036 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21360868-9ff2-4576-824c-524bb1d588fb None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:00.462385 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21360868-9ff2-4576-824c-524bb1d588fb None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:00.462661 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21360868-9ff2-4576-824c-524bb1d588fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:00.497224 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-21360868-9ff2-4576-824c-524bb1d588fb None None] MFA Rules not processed for user `52fc3ba90dc045c2a5b7bb730da96acc`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:28:00.498835 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-21360868-9ff2-4576-824c-524bb1d588fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:00.614263 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1027/2054] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 5 10:28:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:28:00.634508 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76060c57-702e-4459-9c41-7f968e5088aa service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:00.636259 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-76060c57-702e-4459-9c41-7f968e5088aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:00.748385 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76060c57-702e-4459-9c41-7f968e5088aa service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:00.754552 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d5f5d-9b22-44df-ae6b-0ef3a380b34c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c80cf0e429f04f229aea1aa9293a93b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:00.754552 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d5f5d-9b22-44df-ae6b-0ef3a380b34c service swift] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:00.754805 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d5f5d-9b22-44df-ae6b-0ef3a380b34c service swift] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:00.755111 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d5f5d-9b22-44df-ae6b-0ef3a380b34c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:00.766961 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8d5f5d-9b22-44df-ae6b-0ef3a380b34c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:00.769652 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8d5f5d-9b22-44df-ae6b-0ef3a380b34c service swift] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:00.779253 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1028/2055] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 5 10:28:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 05 10:28:00.870365 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb1b70-f4dc-47d3-9072-3f4356e09702 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:00.870836 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb1b70-f4dc-47d3-9072-3f4356e09702 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:00.870966 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb1b70-f4dc-47d3-9072-3f4356e09702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:00.909491 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f1cb1b70-f4dc-47d3-9072-3f4356e09702 None None] MFA Rules not processed for user `7723cdbd378743378b1fccc2d15bc53b`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:28:00.911639 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f1cb1b70-f4dc-47d3-9072-3f4356e09702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:01.010819 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1028/2056] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:28:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:28:01.024608 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d5f5d-9b22-44df-ae6b-0ef3a380b34c service swift] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:01.035476 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d5f5d-9b22-44df-ae6b-0ef3a380b34c service swift] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:01.040938 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af883090-3a66-4d1b-8885-310a9f7f0ee9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:01.042079 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af883090-3a66-4d1b-8885-310a9f7f0ee9 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:01.042537 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af883090-3a66-4d1b-8885-310a9f7f0ee9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:01.043113 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af883090-3a66-4d1b-8885-310a9f7f0ee9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:01.055190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af883090-3a66-4d1b-8885-310a9f7f0ee9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:01.056948 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af883090-3a66-4d1b-8885-310a9f7f0ee9 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:01.080470 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1029/2057] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:28:01 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:28:07.833121 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73853209-32d7-46e7-8372-80ba62c4c1c6 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:07.833830 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73853209-32d7-46e7-8372-80ba62c4c1c6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:07.833830 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73853209-32d7-46e7-8372-80ba62c4c1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:07.853632 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-73853209-32d7-46e7-8372-80ba62c4c1c6 None None] MFA Rules not processed for user `7723cdbd378743378b1fccc2d15bc53b`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:28:07.855070 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-73853209-32d7-46e7-8372-80ba62c4c1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:07.862269 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1029/2058] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:28:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:28:11.063382 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=46 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:28:11.101746 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af883090-3a66-4d1b-8885-310a9f7f0ee9 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:11.113255 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af883090-3a66-4d1b-8885-310a9f7f0ee9 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:11.118272 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5fe8ca-8b27-4ae4-a034-04d1c0a7091f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:11.119122 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5fe8ca-8b27-4ae4-a034-04d1c0a7091f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:11.119501 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5fe8ca-8b27-4ae4-a034-04d1c0a7091f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:11.119881 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5fe8ca-8b27-4ae4-a034-04d1c0a7091f admin admin] PATH_INFO: `/v3/users/1650fd8e00904b8b8925f84e800667db` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:11.121464 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5fe8ca-8b27-4ae4-a034-04d1c0a7091f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1650fd8e00904b8b8925f84e800667db)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:11.123604 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5fe8ca-8b27-4ae4-a034-04d1c0a7091f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:11.191386 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ea5fe8ca-8b27-4ae4-a034-04d1c0a7091f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1650fd8e00904b8b8925f84e800667db'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:11.201808 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ea5fe8ca-8b27-4ae4-a034-04d1c0a7091f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1650fd8e00904b8b8925f84e800667db'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:11.202385 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ea5fe8ca-8b27-4ae4-a034-04d1c0a7091f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1650fd8e00904b8b8925f84e800667db'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:11.216853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ea5fe8ca-8b27-4ae4-a034-04d1c0a7091f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1650fd8e00904b8b8925f84e800667db'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:11.217612 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ea5fe8ca-8b27-4ae4-a034-04d1c0a7091f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1650fd8e00904b8b8925f84e800667db'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:11.230746 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1030/2059] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:28:11 2022] DELETE /identity/v3/users/1650fd8e00904b8b8925f84e800667db => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:28:11.235758 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd50856b-32d4-4a82-9ab0-8d45f1a2f36b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:11.236316 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd50856b-32d4-4a82-9ab0-8d45f1a2f36b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:11.236316 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd50856b-32d4-4a82-9ab0-8d45f1a2f36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:11.250609 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:28:11.257112 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-cd50856b-32d4-4a82-9ab0-8d45f1a2f36b None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:28:11.259398 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cd50856b-32d4-4a82-9ab0-8d45f1a2f36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:11.321159 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1030/2060] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:28:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:28:11.330990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5fe8ca-8b27-4ae4-a034-04d1c0a7091f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:11.332660 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ea5fe8ca-8b27-4ae4-a034-04d1c0a7091f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:11.398234 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5fe8ca-8b27-4ae4-a034-04d1c0a7091f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:11.402627 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8264fc10-ecef-44b7-a291-976a5ecde27f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:11.403292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264fc10-ecef-44b7-a291-976a5ecde27f service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:11.403576 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264fc10-ecef-44b7-a291-976a5ecde27f service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:11.403848 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264fc10-ecef-44b7-a291-976a5ecde27f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:11.414647 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8264fc10-ecef-44b7-a291-976a5ecde27f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:11.416378 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8264fc10-ecef-44b7-a291-976a5ecde27f service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:11.427188 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1031/2061] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:28:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:28:11.760196 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd50856b-32d4-4a82-9ab0-8d45f1a2f36b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:11.761628 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cd50856b-32d4-4a82-9ab0-8d45f1a2f36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:11.774289 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd50856b-32d4-4a82-9ab0-8d45f1a2f36b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:11.779257 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7202488e-3b56-4aa7-82b8-27096cd67982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:11.779986 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7202488e-3b56-4aa7-82b8-27096cd67982 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:11.780338 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7202488e-3b56-4aa7-82b8-27096cd67982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:11.780604 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7202488e-3b56-4aa7-82b8-27096cd67982 admin admin] PATH_INFO: `/v3/projects/516e7abb460f49d3ac19610315c31e92` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:11.783633 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7202488e-3b56-4aa7-82b8-27096cd67982 admin admin] RBAC: Authorizing `identity:delete_project(project_id=516e7abb460f49d3ac19610315c31e92)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:11.785936 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7202488e-3b56-4aa7-82b8-27096cd67982 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:11.871699 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7202488e-3b56-4aa7-82b8-27096cd67982 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '516e7abb460f49d3ac19610315c31e92', 'request_id': 'req-7202488e-3b56-4aa7-82b8-27096cd67982'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:11.911643 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7202488e-3b56-4aa7-82b8-27096cd67982 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '516e7abb460f49d3ac19610315c31e92', 'request_id': 'req-7202488e-3b56-4aa7-82b8-27096cd67982'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:11.918410 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7202488e-3b56-4aa7-82b8-27096cd67982 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '516e7abb460f49d3ac19610315c31e92', 'request_id': 'req-7202488e-3b56-4aa7-82b8-27096cd67982'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:11.931101 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7202488e-3b56-4aa7-82b8-27096cd67982 admin admin] The token cache is being invalidate because project 516e7abb460f49d3ac19610315c31e92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:28:11.931400 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7202488e-3b56-4aa7-82b8-27096cd67982 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:11.932287 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7202488e-3b56-4aa7-82b8-27096cd67982 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:11.934110 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1031/2062] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:28:11 2022] DELETE /identity/v3/projects/516e7abb460f49d3ac19610315c31e92 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:28:11.938496 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8264fc10-ecef-44b7-a291-976a5ecde27f service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:11.950875 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8264fc10-ecef-44b7-a291-976a5ecde27f service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:12.017488 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1392a9a7-603f-4f4e-8c85-29ae8256ea77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:12.018366 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392a9a7-603f-4f4e-8c85-29ae8256ea77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:12.018833 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392a9a7-603f-4f4e-8c85-29ae8256ea77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:12.019211 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392a9a7-603f-4f4e-8c85-29ae8256ea77 admin admin] PATH_INFO: `/v3/users/32e450631cf34293abd406edbe3abb8a` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:12.020627 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1392a9a7-603f-4f4e-8c85-29ae8256ea77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32e450631cf34293abd406edbe3abb8a)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:12.022664 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1392a9a7-603f-4f4e-8c85-29ae8256ea77 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:12.046241 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:28:12.085454 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1392a9a7-603f-4f4e-8c85-29ae8256ea77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32e450631cf34293abd406edbe3abb8a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:12.100785 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1392a9a7-603f-4f4e-8c85-29ae8256ea77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32e450631cf34293abd406edbe3abb8a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:12.103335 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1392a9a7-603f-4f4e-8c85-29ae8256ea77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32e450631cf34293abd406edbe3abb8a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:12.120873 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1392a9a7-603f-4f4e-8c85-29ae8256ea77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32e450631cf34293abd406edbe3abb8a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:12.121621 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1392a9a7-603f-4f4e-8c85-29ae8256ea77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32e450631cf34293abd406edbe3abb8a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:12.139744 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1032/2063] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:28:11 2022] DELETE /identity/v3/users/32e450631cf34293abd406edbe3abb8a => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:28:12.479172 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7202488e-3b56-4aa7-82b8-27096cd67982 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:12.480646 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7202488e-3b56-4aa7-82b8-27096cd67982 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:12.493883 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7202488e-3b56-4aa7-82b8-27096cd67982 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:12.569779 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412effa5-34fe-48f1-b73d-824a59325baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:12.570490 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412effa5-34fe-48f1-b73d-824a59325baa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:12.570765 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412effa5-34fe-48f1-b73d-824a59325baa admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:12.571150 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412effa5-34fe-48f1-b73d-824a59325baa admin admin] PATH_INFO: `/v3/projects/3f25425a02634e8fa3ff7658d226cfbb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:12.572356 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412effa5-34fe-48f1-b73d-824a59325baa admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f25425a02634e8fa3ff7658d226cfbb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:12.574229 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412effa5-34fe-48f1-b73d-824a59325baa admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:12.611195 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:28:12.623811 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-412effa5-34fe-48f1-b73d-824a59325baa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f25425a02634e8fa3ff7658d226cfbb', 'request_id': 'req-412effa5-34fe-48f1-b73d-824a59325baa'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:12.632095 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-412effa5-34fe-48f1-b73d-824a59325baa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f25425a02634e8fa3ff7658d226cfbb', 'request_id': 'req-412effa5-34fe-48f1-b73d-824a59325baa'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:12.638065 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-412effa5-34fe-48f1-b73d-824a59325baa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f25425a02634e8fa3ff7658d226cfbb', 'request_id': 'req-412effa5-34fe-48f1-b73d-824a59325baa'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:12.649642 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-412effa5-34fe-48f1-b73d-824a59325baa admin admin] The token cache is being invalidate because project 3f25425a02634e8fa3ff7658d226cfbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:28:12.649935 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-412effa5-34fe-48f1-b73d-824a59325baa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:12.650688 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-412effa5-34fe-48f1-b73d-824a59325baa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:12.652302 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1032/2064] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:28:12 2022] DELETE /identity/v3/projects/3f25425a02634e8fa3ff7658d226cfbb => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:28:12.657548 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa859cba-c877-42a8-8dfe-2dbcea55d27b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:12.657839 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa859cba-c877-42a8-8dfe-2dbcea55d27b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:12.658103 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa859cba-c877-42a8-8dfe-2dbcea55d27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:12.679399 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-aa859cba-c877-42a8-8dfe-2dbcea55d27b None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:28:12.681249 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-aa859cba-c877-42a8-8dfe-2dbcea55d27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:12.748916 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1033/2065] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:28:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:28:12.757142 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c41c6-e6f9-4da7-a664-120c16270e87 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:12.757407 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c41c6-e6f9-4da7-a664-120c16270e87 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:12.757695 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c41c6-e6f9-4da7-a664-120c16270e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:12.779220 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-531c41c6-e6f9-4da7-a664-120c16270e87 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:28:12.780418 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-531c41c6-e6f9-4da7-a664-120c16270e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:12.791345 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1033/2066] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:28:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:28:12.796058 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa859cba-c877-42a8-8dfe-2dbcea55d27b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:12.806926 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa859cba-c877-42a8-8dfe-2dbcea55d27b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:12.813497 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48113040-c506-4e1c-99c8-bb39b0c7731c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:12.814116 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48113040-c506-4e1c-99c8-bb39b0c7731c admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:12.814396 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48113040-c506-4e1c-99c8-bb39b0c7731c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:12.814665 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48113040-c506-4e1c-99c8-bb39b0c7731c admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:12.815407 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48113040-c506-4e1c-99c8-bb39b0c7731c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:28:12.815702 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48113040-c506-4e1c-99c8-bb39b0c7731c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:12.817283 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48113040-c506-4e1c-99c8-bb39b0c7731c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:12.829668 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1034/2067] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:28:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:28:12.834206 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531c41c6-e6f9-4da7-a664-120c16270e87 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:12.843654 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531c41c6-e6f9-4da7-a664-120c16270e87 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:12.849977 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a4b8af-e085-438a-b62b-b29f611168df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:12.849977 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a4b8af-e085-438a-b62b-b29f611168df admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:12.849977 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a4b8af-e085-438a-b62b-b29f611168df admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:12.849977 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a4b8af-e085-438a-b62b-b29f611168df admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:12.850508 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a4b8af-e085-438a-b62b-b29f611168df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:12.852898 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a4b8af-e085-438a-b62b-b29f611168df admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:12.893767 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1034/2068] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:28:12 2022] POST /identity/v3/projects => generated 371 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:28:12.898620 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48113040-c506-4e1c-99c8-bb39b0c7731c admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:12.916072 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48113040-c506-4e1c-99c8-bb39b0c7731c admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:12.980135 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a2fdf-7d21-4e8e-ab7b-4004840d1873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:12.980760 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a2fdf-7d21-4e8e-ab7b-4004840d1873 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:12.981036 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a2fdf-7d21-4e8e-ab7b-4004840d1873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:12.981300 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a2fdf-7d21-4e8e-ab7b-4004840d1873 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:12.981970 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423a2fdf-7d21-4e8e-ab7b-4004840d1873 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:12.983923 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423a2fdf-7d21-4e8e-ab7b-4004840d1873 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:13.006940 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1035/2069] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:28:12 2022] POST /identity/v3/users => generated 286 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:28:13.011746 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a4b8af-e085-438a-b62b-b29f611168df admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:13.022079 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a4b8af-e085-438a-b62b-b29f611168df admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:13.027347 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4352e9fd-ab88-4d96-8e46-a4fe1e4b79d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:13.027959 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4352e9fd-ab88-4d96-8e46-a4fe1e4b79d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:13.028237 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4352e9fd-ab88-4d96-8e46-a4fe1e4b79d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:13.028506 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4352e9fd-ab88-4d96-8e46-a4fe1e4b79d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:13.029228 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4352e9fd-ab88-4d96-8e46-a4fe1e4b79d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:13.030839 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4352e9fd-ab88-4d96-8e46-a4fe1e4b79d1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:13.041922 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1035/2070] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:28:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:28:13.045946 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a2fdf-7d21-4e8e-ab7b-4004840d1873 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:13.056266 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a2fdf-7d21-4e8e-ab7b-4004840d1873 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:13.060974 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c5616d-49d4-4ee7-b4b1-04eaea15672e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:13.061585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c5616d-49d4-4ee7-b4b1-04eaea15672e admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:13.061865 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c5616d-49d4-4ee7-b4b1-04eaea15672e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:13.062141 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c5616d-49d4-4ee7-b4b1-04eaea15672e admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:13.062836 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c5616d-49d4-4ee7-b4b1-04eaea15672e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:13.064469 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c5616d-49d4-4ee7-b4b1-04eaea15672e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:13.069733 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:28:13.076278 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1036/2071] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:28:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:28:13.080795 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4352e9fd-ab88-4d96-8e46-a4fe1e4b79d1 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:13.093845 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4352e9fd-ab88-4d96-8e46-a4fe1e4b79d1 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:13.099214 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba29d3fb-6ec8-4e56-9a59-6e663f6596fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:13.099874 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29d3fb-6ec8-4e56-9a59-6e663f6596fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:13.100184 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29d3fb-6ec8-4e56-9a59-6e663f6596fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:13.100458 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29d3fb-6ec8-4e56-9a59-6e663f6596fb admin admin] PATH_INFO: `/v3/projects/7ea48942e4bc480d8f1628e825b457c2/users/ab921f405c9b438a805556b81a6dc63d/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:13.124072 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba29d3fb-6ec8-4e56-9a59-6e663f6596fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ea48942e4bc480d8f1628e825b457c2, user_id=ab921f405c9b438a805556b81a6dc63d, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:13.128405 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba29d3fb-6ec8-4e56-9a59-6e663f6596fb admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:13.146371 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1036/2072] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:28:13 2022] PUT /identity/v3/projects/7ea48942e4bc480d8f1628e825b457c2/users/ab921f405c9b438a805556b81a6dc63d/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:28:13.157126 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b097de-1945-4d40-90a3-783b67cb0887 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:13.157701 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b097de-1945-4d40-90a3-783b67cb0887 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:13.157701 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b097de-1945-4d40-90a3-783b67cb0887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:13.190655 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-08b097de-1945-4d40-90a3-783b67cb0887 None None] MFA Rules not processed for user `ab921f405c9b438a805556b81a6dc63d`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:28:13.191896 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-08b097de-1945-4d40-90a3-783b67cb0887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:13.266932 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1037/2073] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:28:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:28:13.281152 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba29d3fb-6ec8-4e56-9a59-6e663f6596fb admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:13.283225 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ba29d3fb-6ec8-4e56-9a59-6e663f6596fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:13.344938 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba29d3fb-6ec8-4e56-9a59-6e663f6596fb admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:13.351489 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde0c63-1ffc-4e59-9b4b-803662a29d76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:13.352253 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde0c63-1ffc-4e59-9b4b-803662a29d76 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:13.352677 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde0c63-1ffc-4e59-9b4b-803662a29d76 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:13.353033 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde0c63-1ffc-4e59-9b4b-803662a29d76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:13.364072 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde0c63-1ffc-4e59-9b4b-803662a29d76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:13.365518 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde0c63-1ffc-4e59-9b4b-803662a29d76 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:13.374890 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1037/2074] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:28:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:28:13.396496 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b097de-1945-4d40-90a3-783b67cb0887 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:13.474454 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b097de-1945-4d40-90a3-783b67cb0887 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:13.481475 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58980b4-8d16-49c1-b350-d01ebb960b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:13.482139 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58980b4-8d16-49c1-b350-d01ebb960b1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:13.482417 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58980b4-8d16-49c1-b350-d01ebb960b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:13.482686 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58980b4-8d16-49c1-b350-d01ebb960b1b admin admin] PATH_INFO: `/v3/users/ab921f405c9b438a805556b81a6dc63d` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:13.483903 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58980b4-8d16-49c1-b350-d01ebb960b1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab921f405c9b438a805556b81a6dc63d)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:13.485868 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58980b4-8d16-49c1-b350-d01ebb960b1b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:13.553977 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c58980b4-8d16-49c1-b350-d01ebb960b1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab921f405c9b438a805556b81a6dc63d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:13.564324 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c58980b4-8d16-49c1-b350-d01ebb960b1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab921f405c9b438a805556b81a6dc63d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:13.565859 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c58980b4-8d16-49c1-b350-d01ebb960b1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab921f405c9b438a805556b81a6dc63d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:13.581567 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c58980b4-8d16-49c1-b350-d01ebb960b1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab921f405c9b438a805556b81a6dc63d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:13.582409 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c58980b4-8d16-49c1-b350-d01ebb960b1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab921f405c9b438a805556b81a6dc63d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:13.596558 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1038/2075] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:28:13 2022] DELETE /identity/v3/users/ab921f405c9b438a805556b81a6dc63d => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:28:13.601899 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec8626-b9d7-400d-adce-9d37e67f31c2 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:13.601899 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec8626-b9d7-400d-adce-9d37e67f31c2 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:13.602636 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec8626-b9d7-400d-adce-9d37e67f31c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:13.626110 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-fbec8626-b9d7-400d-adce-9d37e67f31c2 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:28:13.627542 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fbec8626-b9d7-400d-adce-9d37e67f31c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:13.691401 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:28:13.704099 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1038/2076] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:28:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:28:13.714572 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58980b4-8d16-49c1-b350-d01ebb960b1b admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:13.716139 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c58980b4-8d16-49c1-b350-d01ebb960b1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:13.747409 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:28:13.819998 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58980b4-8d16-49c1-b350-d01ebb960b1b admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:13.823784 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c51ba51-f693-42a6-82c1-a095c61c49b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:13.824421 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51ba51-f693-42a6-82c1-a095c61c49b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:13.824702 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51ba51-f693-42a6-82c1-a095c61c49b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:13.824975 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51ba51-f693-42a6-82c1-a095c61c49b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:13.834855 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c51ba51-f693-42a6-82c1-a095c61c49b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:13.836400 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c51ba51-f693-42a6-82c1-a095c61c49b0 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:13.847056 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1039/2077] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:28:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:28:14.219767 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec8626-b9d7-400d-adce-9d37e67f31c2 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:14.221293 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fbec8626-b9d7-400d-adce-9d37e67f31c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:14.234076 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec8626-b9d7-400d-adce-9d37e67f31c2 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:14.240652 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13665f4-176f-42c7-b6da-5a17bf9fe5f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:14.240652 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13665f4-176f-42c7-b6da-5a17bf9fe5f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:14.240652 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13665f4-176f-42c7-b6da-5a17bf9fe5f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:14.240652 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13665f4-176f-42c7-b6da-5a17bf9fe5f6 admin admin] PATH_INFO: `/v3/projects/7ea48942e4bc480d8f1628e825b457c2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:14.241843 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13665f4-176f-42c7-b6da-5a17bf9fe5f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ea48942e4bc480d8f1628e825b457c2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:14.244135 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13665f4-176f-42c7-b6da-5a17bf9fe5f6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:14.300601 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f13665f4-176f-42c7-b6da-5a17bf9fe5f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ea48942e4bc480d8f1628e825b457c2', 'request_id': 'req-f13665f4-176f-42c7-b6da-5a17bf9fe5f6'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:14.311588 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f13665f4-176f-42c7-b6da-5a17bf9fe5f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ea48942e4bc480d8f1628e825b457c2', 'request_id': 'req-f13665f4-176f-42c7-b6da-5a17bf9fe5f6'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:14.319069 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f13665f4-176f-42c7-b6da-5a17bf9fe5f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ea48942e4bc480d8f1628e825b457c2', 'request_id': 'req-f13665f4-176f-42c7-b6da-5a17bf9fe5f6'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:14.332233 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f13665f4-176f-42c7-b6da-5a17bf9fe5f6 admin admin] The token cache is being invalidate because project 7ea48942e4bc480d8f1628e825b457c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:28:14.332530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f13665f4-176f-42c7-b6da-5a17bf9fe5f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:14.333362 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f13665f4-176f-42c7-b6da-5a17bf9fe5f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:14.338421 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1039/2078] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:28:14 2022] DELETE /identity/v3/projects/7ea48942e4bc480d8f1628e825b457c2 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:28:14.340104 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234669ba-2cc3-4ef7-9ba8-263b875ed8f5 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:14.340694 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234669ba-2cc3-4ef7-9ba8-263b875ed8f5 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:14.340797 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234669ba-2cc3-4ef7-9ba8-263b875ed8f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:14.368854 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-234669ba-2cc3-4ef7-9ba8-263b875ed8f5 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:28:14.370150 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-234669ba-2cc3-4ef7-9ba8-263b875ed8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:14.439777 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1040/2079] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:28:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:28:14.449017 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2553a1-0900-4af8-a755-c2ec9291306f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:14.449303 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2553a1-0900-4af8-a755-c2ec9291306f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:14.449578 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2553a1-0900-4af8-a755-c2ec9291306f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:14.469326 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-5a2553a1-0900-4af8-a755-c2ec9291306f None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:28:14.470538 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5a2553a1-0900-4af8-a755-c2ec9291306f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:14.478504 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1040/2080] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:28:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:28:14.484336 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234669ba-2cc3-4ef7-9ba8-263b875ed8f5 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:14.493985 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234669ba-2cc3-4ef7-9ba8-263b875ed8f5 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:14.498719 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a6f225-b1a3-4374-8608-125711c08b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:14.499375 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6f225-b1a3-4374-8608-125711c08b95 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:14.499660 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6f225-b1a3-4374-8608-125711c08b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:14.499936 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6f225-b1a3-4374-8608-125711c08b95 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:14.500640 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a6f225-b1a3-4374-8608-125711c08b95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:28:14.500947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a6f225-b1a3-4374-8608-125711c08b95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:14.502595 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a6f225-b1a3-4374-8608-125711c08b95 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:14.515162 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1041/2081] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:28:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:28:14.519905 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2553a1-0900-4af8-a755-c2ec9291306f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:14.530335 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2553a1-0900-4af8-a755-c2ec9291306f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:14.536721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5185b591-c72a-4f79-bf36-11d641e3ea3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:14.537550 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5185b591-c72a-4f79-bf36-11d641e3ea3e admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:14.537829 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5185b591-c72a-4f79-bf36-11d641e3ea3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:14.538098 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5185b591-c72a-4f79-bf36-11d641e3ea3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:14.538741 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5185b591-c72a-4f79-bf36-11d641e3ea3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:14.540805 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5185b591-c72a-4f79-bf36-11d641e3ea3e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:14.572445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1041/2082] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:28:14 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:28:14.576778 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a6f225-b1a3-4374-8608-125711c08b95 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:14.587367 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a6f225-b1a3-4374-8608-125711c08b95 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:14.647809 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de812178-9975-4183-bfaf-49c380c916e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:14.648346 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de812178-9975-4183-bfaf-49c380c916e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:14.648619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de812178-9975-4183-bfaf-49c380c916e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:14.648884 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de812178-9975-4183-bfaf-49c380c916e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:14.649672 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de812178-9975-4183-bfaf-49c380c916e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:14.651651 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de812178-9975-4183-bfaf-49c380c916e5 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:14.679918 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1042/2083] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:28:14 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:28:14.681598 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5185b591-c72a-4f79-bf36-11d641e3ea3e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:14.692250 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5185b591-c72a-4f79-bf36-11d641e3ea3e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:14.697579 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c322b-5f68-4afa-82e9-56ba9a2bd24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:14.698200 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c322b-5f68-4afa-82e9-56ba9a2bd24e admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:14.698474 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c322b-5f68-4afa-82e9-56ba9a2bd24e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:14.698740 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c322b-5f68-4afa-82e9-56ba9a2bd24e admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:14.699471 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2c322b-5f68-4afa-82e9-56ba9a2bd24e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:14.701130 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2c322b-5f68-4afa-82e9-56ba9a2bd24e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:14.712361 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1042/2084] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:28:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:28:14.716575 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de812178-9975-4183-bfaf-49c380c916e5 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:14.727183 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de812178-9975-4183-bfaf-49c380c916e5 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:14.732207 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d8387d-c50d-4903-bf9b-637ae45f9208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:14.732841 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8387d-c50d-4903-bf9b-637ae45f9208 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:14.733118 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8387d-c50d-4903-bf9b-637ae45f9208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:14.733387 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8387d-c50d-4903-bf9b-637ae45f9208 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:14.734095 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d8387d-c50d-4903-bf9b-637ae45f9208 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:14.735741 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d8387d-c50d-4903-bf9b-637ae45f9208 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:14.747413 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1043/2085] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:28:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:28:14.751526 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c322b-5f68-4afa-82e9-56ba9a2bd24e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:14.762397 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c322b-5f68-4afa-82e9-56ba9a2bd24e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:14.767428 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448196d1-6ede-4aa9-9707-906fac70b811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:14.768106 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448196d1-6ede-4aa9-9707-906fac70b811 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:14.768383 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448196d1-6ede-4aa9-9707-906fac70b811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:14.768661 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448196d1-6ede-4aa9-9707-906fac70b811 admin admin] PATH_INFO: `/v3/projects/c5f8afe9a8424f63838148b7c59d44e3/users/c2bc5d10c5c84ac0b727d61818cb49fc/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:14.785370 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448196d1-6ede-4aa9-9707-906fac70b811 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5f8afe9a8424f63838148b7c59d44e3, user_id=c2bc5d10c5c84ac0b727d61818cb49fc, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:14.788245 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448196d1-6ede-4aa9-9707-906fac70b811 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:14.819983 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1043/2086] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:28:14 2022] PUT /identity/v3/projects/c5f8afe9a8424f63838148b7c59d44e3/users/c2bc5d10c5c84ac0b727d61818cb49fc/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:28:14.824151 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ada47d-35ed-4197-a9e7-e4511370b9a4 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:14.824437 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ada47d-35ed-4197-a9e7-e4511370b9a4 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:14.824743 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ada47d-35ed-4197-a9e7-e4511370b9a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:14.865386 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-65ada47d-35ed-4197-a9e7-e4511370b9a4 None None] MFA Rules not processed for user `c2bc5d10c5c84ac0b727d61818cb49fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:28:14.867233 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-65ada47d-35ed-4197-a9e7-e4511370b9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:14.873564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:28:14.959487 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1044/2087] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:28:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:28:14.972580 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448196d1-6ede-4aa9-9707-906fac70b811 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:14.974582 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-448196d1-6ede-4aa9-9707-906fac70b811 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:15.037328 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448196d1-6ede-4aa9-9707-906fac70b811 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:15.041989 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1900c1a8-1b81-4f56-bb72-72423b913caa service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac0a8cf7f8b74e88a04444ec38547528', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:15.042611 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900c1a8-1b81-4f56-bb72-72423b913caa service glance] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:15.042916 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900c1a8-1b81-4f56-bb72-72423b913caa service glance] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:15.043197 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900c1a8-1b81-4f56-bb72-72423b913caa service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:15.051408 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1900c1a8-1b81-4f56-bb72-72423b913caa service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:15.052844 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1900c1a8-1b81-4f56-bb72-72423b913caa service glance] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:15.061907 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1044/2088] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:28:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:28:22.055797 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ada47d-35ed-4197-a9e7-e4511370b9a4 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:22.057268 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-65ada47d-35ed-4197-a9e7-e4511370b9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:22.129015 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ada47d-35ed-4197-a9e7-e4511370b9a4 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:22.133817 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee477a-bbcd-414c-ae85-edf839066ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:22.134506 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee477a-bbcd-414c-ae85-edf839066ecd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:22.134790 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee477a-bbcd-414c-ae85-edf839066ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:22.135111 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee477a-bbcd-414c-ae85-edf839066ecd admin admin] PATH_INFO: `/v3/users/1c7b62b1520a4a0cb430479e279e60d8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:22.136314 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ee477a-bbcd-414c-ae85-edf839066ecd admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c7b62b1520a4a0cb430479e279e60d8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:22.138229 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ee477a-bbcd-414c-ae85-edf839066ecd admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:22.225752 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-87ee477a-bbcd-414c-ae85-edf839066ecd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c7b62b1520a4a0cb430479e279e60d8'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:22.234284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-87ee477a-bbcd-414c-ae85-edf839066ecd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c7b62b1520a4a0cb430479e279e60d8'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:22.235081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-87ee477a-bbcd-414c-ae85-edf839066ecd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c7b62b1520a4a0cb430479e279e60d8'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:22.247930 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-87ee477a-bbcd-414c-ae85-edf839066ecd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c7b62b1520a4a0cb430479e279e60d8'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:22.248608 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-87ee477a-bbcd-414c-ae85-edf839066ecd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c7b62b1520a4a0cb430479e279e60d8'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:22.266439 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1045/2089] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:28:22 2022] DELETE /identity/v3/users/1c7b62b1520a4a0cb430479e279e60d8 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:28:22.271025 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797d299-a4bf-43b4-80da-4593060d25e1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:22.271584 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797d299-a4bf-43b4-80da-4593060d25e1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:22.271584 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797d299-a4bf-43b4-80da-4593060d25e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:22.292136 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9797d299-a4bf-43b4-80da-4593060d25e1 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:28:22.293361 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9797d299-a4bf-43b4-80da-4593060d25e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:22.310439 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:28:22.355550 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1045/2090] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:28:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:28:22.366283 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee477a-bbcd-414c-ae85-edf839066ecd admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:22.367840 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-87ee477a-bbcd-414c-ae85-edf839066ecd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:22.412936 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:28:22.442716 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee477a-bbcd-414c-ae85-edf839066ecd admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:22.447602 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddade14b-0729-4cbc-91d5-484e6070cf61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:22.448235 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddade14b-0729-4cbc-91d5-484e6070cf61 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:22.448517 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddade14b-0729-4cbc-91d5-484e6070cf61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:22.448791 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddade14b-0729-4cbc-91d5-484e6070cf61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:22.459848 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddade14b-0729-4cbc-91d5-484e6070cf61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:22.461343 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddade14b-0729-4cbc-91d5-484e6070cf61 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:22.471572 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1046/2091] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:28:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:28:23.000077 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797d299-a4bf-43b4-80da-4593060d25e1 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:23.001887 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9797d299-a4bf-43b4-80da-4593060d25e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:23.018207 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797d299-a4bf-43b4-80da-4593060d25e1 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:23.025892 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98d8fcd-2d42-469c-9018-9b94685bd85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:23.026551 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d8fcd-2d42-469c-9018-9b94685bd85b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:23.026892 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d8fcd-2d42-469c-9018-9b94685bd85b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:23.027125 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d8fcd-2d42-469c-9018-9b94685bd85b admin admin] PATH_INFO: `/v3/projects/ad5593f1bc6d4f448c2ab09a48ac502e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:23.028390 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98d8fcd-2d42-469c-9018-9b94685bd85b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad5593f1bc6d4f448c2ab09a48ac502e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:23.031611 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98d8fcd-2d42-469c-9018-9b94685bd85b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:23.101529 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b98d8fcd-2d42-469c-9018-9b94685bd85b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad5593f1bc6d4f448c2ab09a48ac502e', 'request_id': 'req-b98d8fcd-2d42-469c-9018-9b94685bd85b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:23.116551 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b98d8fcd-2d42-469c-9018-9b94685bd85b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad5593f1bc6d4f448c2ab09a48ac502e', 'request_id': 'req-b98d8fcd-2d42-469c-9018-9b94685bd85b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:23.124851 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b98d8fcd-2d42-469c-9018-9b94685bd85b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad5593f1bc6d4f448c2ab09a48ac502e', 'request_id': 'req-b98d8fcd-2d42-469c-9018-9b94685bd85b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:23.147262 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b98d8fcd-2d42-469c-9018-9b94685bd85b admin admin] The token cache is being invalidate because project ad5593f1bc6d4f448c2ab09a48ac502e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:28:23.147753 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b98d8fcd-2d42-469c-9018-9b94685bd85b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:23.148461 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b98d8fcd-2d42-469c-9018-9b94685bd85b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:23.155425 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1046/2092] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:28:22 2022] DELETE /identity/v3/projects/ad5593f1bc6d4f448c2ab09a48ac502e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:28:32.466938 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=14 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:28:33.142551 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=44,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:28:43.586414 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddade14b-0729-4cbc-91d5-484e6070cf61 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:43.587969 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ddade14b-0729-4cbc-91d5-484e6070cf61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:43.601271 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddade14b-0729-4cbc-91d5-484e6070cf61 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:43.662720 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde6cdfd-7469-455b-9559-b9013cf6384e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:43.663449 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6cdfd-7469-455b-9559-b9013cf6384e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:43.663734 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6cdfd-7469-455b-9559-b9013cf6384e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:43.664008 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6cdfd-7469-455b-9559-b9013cf6384e admin admin] PATH_INFO: `/v3/users/c2bc5d10c5c84ac0b727d61818cb49fc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:43.665200 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde6cdfd-7469-455b-9559-b9013cf6384e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2bc5d10c5c84ac0b727d61818cb49fc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:43.667158 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde6cdfd-7469-455b-9559-b9013cf6384e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:43.720836 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dde6cdfd-7469-455b-9559-b9013cf6384e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2bc5d10c5c84ac0b727d61818cb49fc'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:43.730797 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dde6cdfd-7469-455b-9559-b9013cf6384e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2bc5d10c5c84ac0b727d61818cb49fc'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:43.731745 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dde6cdfd-7469-455b-9559-b9013cf6384e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2bc5d10c5c84ac0b727d61818cb49fc'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:43.744900 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dde6cdfd-7469-455b-9559-b9013cf6384e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2bc5d10c5c84ac0b727d61818cb49fc'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:43.745575 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dde6cdfd-7469-455b-9559-b9013cf6384e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2bc5d10c5c84ac0b727d61818cb49fc'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:43.757739 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1047/2093] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:28:43 2022] DELETE /identity/v3/users/c2bc5d10c5c84ac0b727d61818cb49fc => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:28:43.762238 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f3938-b182-4420-8a57-9fb14141212f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:43.762731 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f3938-b182-4420-8a57-9fb14141212f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:43.762924 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f3938-b182-4420-8a57-9fb14141212f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:43.783333 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-fc4f3938-b182-4420-8a57-9fb14141212f None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:28:43.784957 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fc4f3938-b182-4420-8a57-9fb14141212f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:43.849478 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1047/2094] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:28:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:28:43.859574 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde6cdfd-7469-455b-9559-b9013cf6384e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:43.861095 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-dde6cdfd-7469-455b-9559-b9013cf6384e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:43.923195 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde6cdfd-7469-455b-9559-b9013cf6384e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:43.927756 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e570e-4b1b-4164-9f00-ebffd9bb9ebb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:43.928370 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e570e-4b1b-4164-9f00-ebffd9bb9ebb service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:43.928645 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e570e-4b1b-4164-9f00-ebffd9bb9ebb service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:43.928913 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e570e-4b1b-4164-9f00-ebffd9bb9ebb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:43.938813 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417e570e-4b1b-4164-9f00-ebffd9bb9ebb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:43.940281 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417e570e-4b1b-4164-9f00-ebffd9bb9ebb service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:43.950264 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1048/2095] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:28:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:28:44.244564 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4f3938-b182-4420-8a57-9fb14141212f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:44.246220 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fc4f3938-b182-4420-8a57-9fb14141212f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:44.261866 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4f3938-b182-4420-8a57-9fb14141212f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:44.267356 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1dadcb-3eed-405a-9092-daadaf3c05e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:44.267940 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1dadcb-3eed-405a-9092-daadaf3c05e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:44.268343 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1dadcb-3eed-405a-9092-daadaf3c05e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:44.268683 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1dadcb-3eed-405a-9092-daadaf3c05e0 admin admin] PATH_INFO: `/v3/projects/c5f8afe9a8424f63838148b7c59d44e3` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:44.269944 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1dadcb-3eed-405a-9092-daadaf3c05e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5f8afe9a8424f63838148b7c59d44e3)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:44.271910 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1dadcb-3eed-405a-9092-daadaf3c05e0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:44.324731 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-bc1dadcb-3eed-405a-9092-daadaf3c05e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5f8afe9a8424f63838148b7c59d44e3', 'request_id': 'req-bc1dadcb-3eed-405a-9092-daadaf3c05e0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:44.334802 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-bc1dadcb-3eed-405a-9092-daadaf3c05e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5f8afe9a8424f63838148b7c59d44e3', 'request_id': 'req-bc1dadcb-3eed-405a-9092-daadaf3c05e0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:44.340754 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-bc1dadcb-3eed-405a-9092-daadaf3c05e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5f8afe9a8424f63838148b7c59d44e3', 'request_id': 'req-bc1dadcb-3eed-405a-9092-daadaf3c05e0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:44.352769 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-bc1dadcb-3eed-405a-9092-daadaf3c05e0 admin admin] The token cache is being invalidate because project c5f8afe9a8424f63838148b7c59d44e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:28:44.353059 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-bc1dadcb-3eed-405a-9092-daadaf3c05e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:44.353783 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-bc1dadcb-3eed-405a-9092-daadaf3c05e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:44.355478 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1048/2096] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:28:44 2022] DELETE /identity/v3/projects/c5f8afe9a8424f63838148b7c59d44e3 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:28:44.360678 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d48b9b-9a5d-4272-a867-aaddb4e391a9 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:44.360868 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d48b9b-9a5d-4272-a867-aaddb4e391a9 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:44.361166 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d48b9b-9a5d-4272-a867-aaddb4e391a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:44.382239 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-73d48b9b-9a5d-4272-a867-aaddb4e391a9 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:28:44.383461 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-73d48b9b-9a5d-4272-a867-aaddb4e391a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:44.402694 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:28:44.449721 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1049/2097] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:28:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:28:44.459239 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54421f97-198f-4c4c-9b06-f8e8345207fc None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:44.459528 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54421f97-198f-4c4c-9b06-f8e8345207fc None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:44.459803 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54421f97-198f-4c4c-9b06-f8e8345207fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:44.480568 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-54421f97-198f-4c4c-9b06-f8e8345207fc None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:28:44.481783 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-54421f97-198f-4c4c-9b06-f8e8345207fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:44.491643 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1049/2098] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:28:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:28:44.495848 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d48b9b-9a5d-4272-a867-aaddb4e391a9 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:44.506140 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d48b9b-9a5d-4272-a867-aaddb4e391a9 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:44.511441 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bcee03-ffe5-4383-8d3f-09651e3bf4a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:44.512058 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bcee03-ffe5-4383-8d3f-09651e3bf4a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:44.512341 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bcee03-ffe5-4383-8d3f-09651e3bf4a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:44.512615 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bcee03-ffe5-4383-8d3f-09651e3bf4a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:44.513849 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bcee03-ffe5-4383-8d3f-09651e3bf4a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:28:44.513849 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bcee03-ffe5-4383-8d3f-09651e3bf4a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:44.515241 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bcee03-ffe5-4383-8d3f-09651e3bf4a3 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:44.527412 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1050/2099] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:28:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:28:44.531924 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54421f97-198f-4c4c-9b06-f8e8345207fc None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:44.543217 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54421f97-198f-4c4c-9b06-f8e8345207fc None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:44.548738 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22114b7a-74ce-4eb4-a340-684593b35ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:44.549353 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22114b7a-74ce-4eb4-a340-684593b35ce9 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:44.549627 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22114b7a-74ce-4eb4-a340-684593b35ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:44.550169 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22114b7a-74ce-4eb4-a340-684593b35ce9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:44.550752 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22114b7a-74ce-4eb4-a340-684593b35ce9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:44.552892 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22114b7a-74ce-4eb4-a340-684593b35ce9 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:44.589867 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1050/2100] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:28:44 2022] POST /identity/v3/projects => generated 375 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:28:44.593491 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bcee03-ffe5-4383-8d3f-09651e3bf4a3 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:44.605510 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bcee03-ffe5-4383-8d3f-09651e3bf4a3 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:44.682954 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e768d58f-4ff6-4d36-b35d-2f5ff894dcf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:44.683606 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e768d58f-4ff6-4d36-b35d-2f5ff894dcf9 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:44.683884 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e768d58f-4ff6-4d36-b35d-2f5ff894dcf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:44.684150 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e768d58f-4ff6-4d36-b35d-2f5ff894dcf9 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:44.684805 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e768d58f-4ff6-4d36-b35d-2f5ff894dcf9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:44.686718 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e768d58f-4ff6-4d36-b35d-2f5ff894dcf9 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:44.709852 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1051/2101] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:28:44 2022] POST /identity/v3/users => generated 288 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:28:44.713815 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22114b7a-74ce-4eb4-a340-684593b35ce9 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:44.725032 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22114b7a-74ce-4eb4-a340-684593b35ce9 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:44.729826 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92db30b5-14bf-4cc7-b03d-92f4dedc8d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:44.730433 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db30b5-14bf-4cc7-b03d-92f4dedc8d73 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:44.730682 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db30b5-14bf-4cc7-b03d-92f4dedc8d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:44.730991 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db30b5-14bf-4cc7-b03d-92f4dedc8d73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:44.731680 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92db30b5-14bf-4cc7-b03d-92f4dedc8d73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:44.733289 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92db30b5-14bf-4cc7-b03d-92f4dedc8d73 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:44.744100 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1051/2102] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:28:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:28:44.748232 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e768d58f-4ff6-4d36-b35d-2f5ff894dcf9 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:44.759046 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e768d58f-4ff6-4d36-b35d-2f5ff894dcf9 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:44.765254 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ad171-7c7a-42f3-a3aa-721dd07f0190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:44.765868 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ad171-7c7a-42f3-a3aa-721dd07f0190 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:44.766155 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ad171-7c7a-42f3-a3aa-721dd07f0190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:44.766445 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ad171-7c7a-42f3-a3aa-721dd07f0190 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:44.767180 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ad171-7c7a-42f3-a3aa-721dd07f0190 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:44.768884 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ad171-7c7a-42f3-a3aa-721dd07f0190 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:44.780650 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1052/2103] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:28:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:28:44.787211 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92db30b5-14bf-4cc7-b03d-92f4dedc8d73 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:44.802556 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92db30b5-14bf-4cc7-b03d-92f4dedc8d73 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:44.808262 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615f4910-4940-4076-a2e4-8c0555da07a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:44.808953 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f4910-4940-4076-a2e4-8c0555da07a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:44.809230 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f4910-4940-4076-a2e4-8c0555da07a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:44.809498 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f4910-4940-4076-a2e4-8c0555da07a6 admin admin] PATH_INFO: `/v3/projects/7e4c6279c9654853a6ddaa563fdd0617/users/0e181d2b1f7142479079dd829673f910/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:44.825235 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615f4910-4940-4076-a2e4-8c0555da07a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e4c6279c9654853a6ddaa563fdd0617, user_id=0e181d2b1f7142479079dd829673f910, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:44.827972 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615f4910-4940-4076-a2e4-8c0555da07a6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:44.843213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1052/2104] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:28:44 2022] PUT /identity/v3/projects/7e4c6279c9654853a6ddaa563fdd0617/users/0e181d2b1f7142479079dd829673f910/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:28:44.853649 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa7cc9-e270-4a82-8bbe-d1b5b8b2fafa None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:44.854082 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa7cc9-e270-4a82-8bbe-d1b5b8b2fafa None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:44.854497 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa7cc9-e270-4a82-8bbe-d1b5b8b2fafa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:44.883497 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-fafa7cc9-e270-4a82-8bbe-d1b5b8b2fafa None None] MFA Rules not processed for user `0e181d2b1f7142479079dd829673f910`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:28:44.884606 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fafa7cc9-e270-4a82-8bbe-d1b5b8b2fafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:44.951465 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1053/2105] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:28:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:28:44.956334 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615f4910-4940-4076-a2e4-8c0555da07a6 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:44.979715 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:28:45.023855 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615f4910-4940-4076-a2e4-8c0555da07a6 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:45.028724 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fcc2c6-6d76-4574-9945-7ae90cce526a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:45.029344 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fcc2c6-6d76-4574-9945-7ae90cce526a admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:45.029631 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fcc2c6-6d76-4574-9945-7ae90cce526a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:45.029943 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fcc2c6-6d76-4574-9945-7ae90cce526a admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:45.030623 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fcc2c6-6d76-4574-9945-7ae90cce526a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:45.032572 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fcc2c6-6d76-4574-9945-7ae90cce526a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:45.067870 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1053/2106] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:28:44 2022] POST /identity/v3/projects => generated 375 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:28:45.072045 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa7cc9-e270-4a82-8bbe-d1b5b8b2fafa None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:45.083389 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa7cc9-e270-4a82-8bbe-d1b5b8b2fafa None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:45.096279 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:28:45.143455 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943133aa-6a5c-4be6-ba06-3ae958598e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:45.144019 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943133aa-6a5c-4be6-ba06-3ae958598e93 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:45.144299 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943133aa-6a5c-4be6-ba06-3ae958598e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:45.144559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943133aa-6a5c-4be6-ba06-3ae958598e93 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:45.145234 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943133aa-6a5c-4be6-ba06-3ae958598e93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:45.147185 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943133aa-6a5c-4be6-ba06-3ae958598e93 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:45.175894 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1054/2107] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:28:45 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:28:45.180010 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fcc2c6-6d76-4574-9945-7ae90cce526a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:45.190417 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fcc2c6-6d76-4574-9945-7ae90cce526a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:45.195329 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11568cf7-582b-40eb-9979-11acce01dace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:45.195952 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11568cf7-582b-40eb-9979-11acce01dace admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:45.196235 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11568cf7-582b-40eb-9979-11acce01dace admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:45.196509 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11568cf7-582b-40eb-9979-11acce01dace admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:45.197205 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11568cf7-582b-40eb-9979-11acce01dace admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:45.198957 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11568cf7-582b-40eb-9979-11acce01dace admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:45.209827 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1054/2108] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:28:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:28:45.214137 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943133aa-6a5c-4be6-ba06-3ae958598e93 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:45.224677 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943133aa-6a5c-4be6-ba06-3ae958598e93 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:45.230153 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c487f449-bed8-4813-af4c-8f3d6ef1d90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:45.230815 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c487f449-bed8-4813-af4c-8f3d6ef1d90a admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:45.231129 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c487f449-bed8-4813-af4c-8f3d6ef1d90a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:45.231405 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c487f449-bed8-4813-af4c-8f3d6ef1d90a admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:45.232109 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c487f449-bed8-4813-af4c-8f3d6ef1d90a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:45.233699 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c487f449-bed8-4813-af4c-8f3d6ef1d90a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:45.244196 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1055/2109] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:28:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:28:45.248422 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11568cf7-582b-40eb-9979-11acce01dace admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:45.258766 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11568cf7-582b-40eb-9979-11acce01dace admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:45.263670 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5c6bc-50d7-45f5-af53-f1fda393974c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:45.264333 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5c6bc-50d7-45f5-af53-f1fda393974c admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:45.264612 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5c6bc-50d7-45f5-af53-f1fda393974c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:45.264886 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5c6bc-50d7-45f5-af53-f1fda393974c admin admin] PATH_INFO: `/v3/projects/10e0d1a2253742f5aed69a5d1e834084/users/e9af3300d5b6442b9dc5cf8fb0d6d15b/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:45.279732 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d5c6bc-50d7-45f5-af53-f1fda393974c admin admin] RBAC: Authorizing `identity:create_grant(project_id=10e0d1a2253742f5aed69a5d1e834084, user_id=e9af3300d5b6442b9dc5cf8fb0d6d15b, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:45.282389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d5c6bc-50d7-45f5-af53-f1fda393974c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:45.300854 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1055/2110] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:28:45 2022] PUT /identity/v3/projects/10e0d1a2253742f5aed69a5d1e834084/users/e9af3300d5b6442b9dc5cf8fb0d6d15b/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:28:45.311333 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b86a42-90da-45d8-a5c4-fc2d843a2c49 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:45.311623 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b86a42-90da-45d8-a5c4-fc2d843a2c49 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:45.311898 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b86a42-90da-45d8-a5c4-fc2d843a2c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:45.340927 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-13b86a42-90da-45d8-a5c4-fc2d843a2c49 None None] MFA Rules not processed for user `e9af3300d5b6442b9dc5cf8fb0d6d15b`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:28:45.342143 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-13b86a42-90da-45d8-a5c4-fc2d843a2c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:45.420723 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1056/2111] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:28:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:28:45.425780 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5c6bc-50d7-45f5-af53-f1fda393974c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:45.491974 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5c6bc-50d7-45f5-af53-f1fda393974c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:45.497289 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51080a9a-1cef-4bf4-8952-8338b15c61b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:45.497907 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51080a9a-1cef-4bf4-8952-8338b15c61b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:45.498187 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51080a9a-1cef-4bf4-8952-8338b15c61b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:45.498458 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51080a9a-1cef-4bf4-8952-8338b15c61b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:45.499153 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51080a9a-1cef-4bf4-8952-8338b15c61b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:45.501083 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51080a9a-1cef-4bf4-8952-8338b15c61b7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:45.531739 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1056/2112] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:28:45 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:28:45.536411 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b86a42-90da-45d8-a5c4-fc2d843a2c49 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:45.547373 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b86a42-90da-45d8-a5c4-fc2d843a2c49 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:45.605083 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba50122-9919-4700-b268-25b710abccfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:45.605783 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba50122-9919-4700-b268-25b710abccfc admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:45.605968 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba50122-9919-4700-b268-25b710abccfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:45.606241 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba50122-9919-4700-b268-25b710abccfc admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:45.606929 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba50122-9919-4700-b268-25b710abccfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:45.608968 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba50122-9919-4700-b268-25b710abccfc admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:45.631846 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1057/2113] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:28:45 2022] POST /identity/v3/users => generated 295 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:28:45.635880 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51080a9a-1cef-4bf4-8952-8338b15c61b7 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:45.646693 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51080a9a-1cef-4bf4-8952-8338b15c61b7 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:45.652472 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc4a97f-d3df-4502-8bf1-d7a713d058a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:45.653353 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc4a97f-d3df-4502-8bf1-d7a713d058a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:45.653627 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc4a97f-d3df-4502-8bf1-d7a713d058a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:45.653896 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc4a97f-d3df-4502-8bf1-d7a713d058a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:45.654579 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc4a97f-d3df-4502-8bf1-d7a713d058a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:45.656202 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc4a97f-d3df-4502-8bf1-d7a713d058a7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:45.668292 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1057/2114] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:28:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 10:28:45.672574 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba50122-9919-4700-b268-25b710abccfc admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:45.684705 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba50122-9919-4700-b268-25b710abccfc admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:45.690986 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c77ed6a-79cb-44dd-afed-5bfd0a45f4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:45.691673 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77ed6a-79cb-44dd-afed-5bfd0a45f4c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:45.691953 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77ed6a-79cb-44dd-afed-5bfd0a45f4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:45.692293 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77ed6a-79cb-44dd-afed-5bfd0a45f4c2 admin admin] PATH_INFO: `/v3/projects/6b08d9c9534f495d9df4245c3fe7521c/users/835de29e84ca4e269c9b412781b1c900/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:45.718678 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:28:45.722732 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c77ed6a-79cb-44dd-afed-5bfd0a45f4c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b08d9c9534f495d9df4245c3fe7521c, user_id=835de29e84ca4e269c9b412781b1c900, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:45.726176 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c77ed6a-79cb-44dd-afed-5bfd0a45f4c2 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:45.749265 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1058/2115] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:28:45 2022] PUT /identity/v3/projects/6b08d9c9534f495d9df4245c3fe7521c/users/835de29e84ca4e269c9b412781b1c900/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:28:45.753210 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc4a97f-d3df-4502-8bf1-d7a713d058a7 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:45.764494 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc4a97f-d3df-4502-8bf1-d7a713d058a7 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:45.824461 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:28:45.861004 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b29045-0a90-43d8-9dcd-718e98d3ab3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:45.861697 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b29045-0a90-43d8-9dcd-718e98d3ab3d admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:45.861979 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b29045-0a90-43d8-9dcd-718e98d3ab3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:45.862293 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b29045-0a90-43d8-9dcd-718e98d3ab3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:45.863535 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b29045-0a90-43d8-9dcd-718e98d3ab3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:45.865149 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b29045-0a90-43d8-9dcd-718e98d3ab3d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:45.875793 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1058/2116] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:28:45 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:28:45.880366 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c77ed6a-79cb-44dd-afed-5bfd0a45f4c2 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:45.891152 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c77ed6a-79cb-44dd-afed-5bfd0a45f4c2 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:45.895902 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b91ad6-cb10-4d33-aad1-b4ab933f12e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:45.896636 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b91ad6-cb10-4d33-aad1-b4ab933f12e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:45.896891 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b91ad6-cb10-4d33-aad1-b4ab933f12e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:45.897192 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b91ad6-cb10-4d33-aad1-b4ab933f12e4 admin admin] PATH_INFO: `/v3/projects/6b08d9c9534f495d9df4245c3fe7521c/users/835de29e84ca4e269c9b412781b1c900/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:45.899468 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b91ad6-cb10-4d33-aad1-b4ab933f12e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b08d9c9534f495d9df4245c3fe7521c, user_id=835de29e84ca4e269c9b412781b1c900, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:45.902102 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b91ad6-cb10-4d33-aad1-b4ab933f12e4 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:45.913499 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1059/2117] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:28:45 2022] PUT /identity/v3/projects/6b08d9c9534f495d9df4245c3fe7521c/users/835de29e84ca4e269c9b412781b1c900/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:28:45.925310 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46faa86-0128-4663-955e-01a8ce1d7b92 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:45.925526 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46faa86-0128-4663-955e-01a8ce1d7b92 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:45.925807 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46faa86-0128-4663-955e-01a8ce1d7b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:45.953949 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a46faa86-0128-4663-955e-01a8ce1d7b92 None None] MFA Rules not processed for user `835de29e84ca4e269c9b412781b1c900`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:28:45.955165 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a46faa86-0128-4663-955e-01a8ce1d7b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:46.035009 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1059/2118] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:28:45 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:28:46.042971 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b91ad6-cb10-4d33-aad1-b4ab933f12e4 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:46.044896 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b6b91ad6-cb10-4d33-aad1-b4ab933f12e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:46.109064 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b91ad6-cb10-4d33-aad1-b4ab933f12e4 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:46.112877 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d254da-43f6-4d29-9c13-64a3f16dc757 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:46.113505 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d254da-43f6-4d29-9c13-64a3f16dc757 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:46.114280 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d254da-43f6-4d29-9c13-64a3f16dc757 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:46.114280 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d254da-43f6-4d29-9c13-64a3f16dc757 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:46.178061 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d254da-43f6-4d29-9c13-64a3f16dc757 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:46.179627 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d254da-43f6-4d29-9c13-64a3f16dc757 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:46.190563 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1060/2119] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:28:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:28:47.749251 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46faa86-0128-4663-955e-01a8ce1d7b92 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:47.764017 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46faa86-0128-4663-955e-01a8ce1d7b92 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:47.768665 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28cef90-3536-42d2-9b18-1495159c5797 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:47.770471 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28cef90-3536-42d2-9b18-1495159c5797 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:47.770742 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28cef90-3536-42d2-9b18-1495159c5797 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:47.771045 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28cef90-3536-42d2-9b18-1495159c5797 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:47.832224 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28cef90-3536-42d2-9b18-1495159c5797 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:47.833698 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28cef90-3536-42d2-9b18-1495159c5797 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:47.844088 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1060/2120] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:28:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:28:48.463406 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d254da-43f6-4d29-9c13-64a3f16dc757 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:48.480911 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d254da-43f6-4d29-9c13-64a3f16dc757 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:48.484586 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e3f727-61eb-4b7c-aaa1-c01c49c251e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:48.485221 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e3f727-61eb-4b7c-aaa1-c01c49c251e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:48.485502 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e3f727-61eb-4b7c-aaa1-c01c49c251e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:48.485775 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e3f727-61eb-4b7c-aaa1-c01c49c251e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:48.496663 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e3f727-61eb-4b7c-aaa1-c01c49c251e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:48.498668 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e3f727-61eb-4b7c-aaa1-c01c49c251e4 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:48.509472 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1061/2121] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:28:48 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:28:49.581632 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb009c-d41a-4462-87eb-43b54e0b2020 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:49.582147 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb009c-d41a-4462-87eb-43b54e0b2020 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:49.582285 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb009c-d41a-4462-87eb-43b54e0b2020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:49.601520 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-cfcb009c-d41a-4462-87eb-43b54e0b2020 None None] MFA Rules not processed for user `7723cdbd378743378b1fccc2d15bc53b`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:28:49.602732 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cfcb009c-d41a-4462-87eb-43b54e0b2020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:49.661963 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1061/2122] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:28:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:28:52.425409 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e3f727-61eb-4b7c-aaa1-c01c49c251e4 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:52.493654 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e3f727-61eb-4b7c-aaa1-c01c49c251e4 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:52.498402 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb7e6a2-ab7e-4685-9d56-55113c033680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:52.499098 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb7e6a2-ab7e-4685-9d56-55113c033680 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:52.499383 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb7e6a2-ab7e-4685-9d56-55113c033680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:52.499660 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb7e6a2-ab7e-4685-9d56-55113c033680 admin admin] PATH_INFO: `/v3/users/0e181d2b1f7142479079dd829673f910` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:52.500905 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb7e6a2-ab7e-4685-9d56-55113c033680 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e181d2b1f7142479079dd829673f910)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:52.502859 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb7e6a2-ab7e-4685-9d56-55113c033680 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:52.531258 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:28:52.555574 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5cb7e6a2-ab7e-4685-9d56-55113c033680 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e181d2b1f7142479079dd829673f910'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:52.564565 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5cb7e6a2-ab7e-4685-9d56-55113c033680 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e181d2b1f7142479079dd829673f910'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:52.565279 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5cb7e6a2-ab7e-4685-9d56-55113c033680 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e181d2b1f7142479079dd829673f910'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:52.578162 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5cb7e6a2-ab7e-4685-9d56-55113c033680 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e181d2b1f7142479079dd829673f910'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:52.578849 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5cb7e6a2-ab7e-4685-9d56-55113c033680 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e181d2b1f7142479079dd829673f910'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:52.591333 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1062/2123] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:28:52 2022] DELETE /identity/v3/users/0e181d2b1f7142479079dd829673f910 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:28:52.596086 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584cc3b3-7071-4266-b8cc-17d47bfe4584 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:52.596491 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584cc3b3-7071-4266-b8cc-17d47bfe4584 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:52.596623 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584cc3b3-7071-4266-b8cc-17d47bfe4584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:52.616570 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-584cc3b3-7071-4266-b8cc-17d47bfe4584 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:28:52.617909 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-584cc3b3-7071-4266-b8cc-17d47bfe4584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:52.622389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:28:52.675174 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb7e6a2-ab7e-4685-9d56-55113c033680 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:52.676494 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5cb7e6a2-ab7e-4685-9d56-55113c033680 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:52.684510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1062/2124] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:28:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:28:52.695335 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584cc3b3-7071-4266-b8cc-17d47bfe4584 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:52.696993 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-584cc3b3-7071-4266-b8cc-17d47bfe4584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:52.748809 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb7e6a2-ab7e-4685-9d56-55113c033680 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:52.752640 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8a3a0c-85ce-4988-80d1-5e55578a8555 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:52.753146 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a3a0c-85ce-4988-80d1-5e55578a8555 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:52.753419 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a3a0c-85ce-4988-80d1-5e55578a8555 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:52.753687 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a3a0c-85ce-4988-80d1-5e55578a8555 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:52.755735 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ba8a3a0c-85ce-4988-80d1-5e55578a8555 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:52.766051 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584cc3b3-7071-4266-b8cc-17d47bfe4584 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:52.771569 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c5349-856a-442e-ad06-1a589c394a27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:52.772497 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c5349-856a-442e-ad06-1a589c394a27 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:52.772951 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c5349-856a-442e-ad06-1a589c394a27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:52.773531 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c5349-856a-442e-ad06-1a589c394a27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:52.784797 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1c5349-856a-442e-ad06-1a589c394a27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:52.786947 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1c5349-856a-442e-ad06-1a589c394a27 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:52.798346 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1063/2125] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:28:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 10:28:52.831448 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8a3a0c-85ce-4988-80d1-5e55578a8555 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:52.833177 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8a3a0c-85ce-4988-80d1-5e55578a8555 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:52.849486 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1063/2126] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:28:52 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:28:53.120624 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c5349-856a-442e-ad06-1a589c394a27 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:53.122216 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3b1c5349-856a-442e-ad06-1a589c394a27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:53.165555 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c5349-856a-442e-ad06-1a589c394a27 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:53.174824 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf55f699-f234-4591-b2e0-8ce7f13ef644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:53.176320 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55f699-f234-4591-b2e0-8ce7f13ef644 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:53.176610 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55f699-f234-4591-b2e0-8ce7f13ef644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:53.176914 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55f699-f234-4591-b2e0-8ce7f13ef644 admin admin] PATH_INFO: `/v3/projects/7e4c6279c9654853a6ddaa563fdd0617` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:53.178789 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf55f699-f234-4591-b2e0-8ce7f13ef644 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e4c6279c9654853a6ddaa563fdd0617)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:53.181302 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf55f699-f234-4591-b2e0-8ce7f13ef644 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:53.245897 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cf55f699-f234-4591-b2e0-8ce7f13ef644 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e4c6279c9654853a6ddaa563fdd0617', 'request_id': 'req-cf55f699-f234-4591-b2e0-8ce7f13ef644'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:53.257356 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cf55f699-f234-4591-b2e0-8ce7f13ef644 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e4c6279c9654853a6ddaa563fdd0617', 'request_id': 'req-cf55f699-f234-4591-b2e0-8ce7f13ef644'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:53.264982 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cf55f699-f234-4591-b2e0-8ce7f13ef644 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e4c6279c9654853a6ddaa563fdd0617', 'request_id': 'req-cf55f699-f234-4591-b2e0-8ce7f13ef644'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:53.281840 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cf55f699-f234-4591-b2e0-8ce7f13ef644 admin admin] The token cache is being invalidate because project 7e4c6279c9654853a6ddaa563fdd0617 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:28:53.282938 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cf55f699-f234-4591-b2e0-8ce7f13ef644 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:53.283080 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cf55f699-f234-4591-b2e0-8ce7f13ef644 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:53.289120 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1064/2127] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:28:53 2022] DELETE /identity/v3/projects/7e4c6279c9654853a6ddaa563fdd0617 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:28:53.290037 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8a3a0c-85ce-4988-80d1-5e55578a8555 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:53.304659 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8a3a0c-85ce-4988-80d1-5e55578a8555 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:53.360670 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:28:53.374711 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee014d-932f-4bdd-b13d-dd4d8f6e9c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:53.375414 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee014d-932f-4bdd-b13d-dd4d8f6e9c60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:53.375696 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee014d-932f-4bdd-b13d-dd4d8f6e9c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:53.375971 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee014d-932f-4bdd-b13d-dd4d8f6e9c60 admin admin] PATH_INFO: `/v3/users/e9af3300d5b6442b9dc5cf8fb0d6d15b` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:53.377139 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ee014d-932f-4bdd-b13d-dd4d8f6e9c60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9af3300d5b6442b9dc5cf8fb0d6d15b)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:53.379142 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ee014d-932f-4bdd-b13d-dd4d8f6e9c60 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:53.428483 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-39ee014d-932f-4bdd-b13d-dd4d8f6e9c60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9af3300d5b6442b9dc5cf8fb0d6d15b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:53.438202 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-39ee014d-932f-4bdd-b13d-dd4d8f6e9c60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9af3300d5b6442b9dc5cf8fb0d6d15b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:53.439483 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-39ee014d-932f-4bdd-b13d-dd4d8f6e9c60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9af3300d5b6442b9dc5cf8fb0d6d15b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:53.452581 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-39ee014d-932f-4bdd-b13d-dd4d8f6e9c60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9af3300d5b6442b9dc5cf8fb0d6d15b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:53.453307 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-39ee014d-932f-4bdd-b13d-dd4d8f6e9c60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9af3300d5b6442b9dc5cf8fb0d6d15b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:53.466158 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1064/2128] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:28:53 2022] DELETE /identity/v3/users/e9af3300d5b6442b9dc5cf8fb0d6d15b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:28:53.757605 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf55f699-f234-4591-b2e0-8ce7f13ef644 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:53.759294 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cf55f699-f234-4591-b2e0-8ce7f13ef644 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:53.776302 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf55f699-f234-4591-b2e0-8ce7f13ef644 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:53.824506 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:28:53.851771 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84a2e45-8957-42bc-b2f4-1ea1db339285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:53.852429 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84a2e45-8957-42bc-b2f4-1ea1db339285 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:53.852708 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84a2e45-8957-42bc-b2f4-1ea1db339285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:53.852982 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84a2e45-8957-42bc-b2f4-1ea1db339285 admin admin] PATH_INFO: `/v3/projects/10e0d1a2253742f5aed69a5d1e834084` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:53.854171 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84a2e45-8957-42bc-b2f4-1ea1db339285 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10e0d1a2253742f5aed69a5d1e834084)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:53.856126 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84a2e45-8957-42bc-b2f4-1ea1db339285 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:53.920335 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f84a2e45-8957-42bc-b2f4-1ea1db339285 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10e0d1a2253742f5aed69a5d1e834084', 'request_id': 'req-f84a2e45-8957-42bc-b2f4-1ea1db339285'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:53.929133 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f84a2e45-8957-42bc-b2f4-1ea1db339285 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10e0d1a2253742f5aed69a5d1e834084', 'request_id': 'req-f84a2e45-8957-42bc-b2f4-1ea1db339285'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:53.935569 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f84a2e45-8957-42bc-b2f4-1ea1db339285 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10e0d1a2253742f5aed69a5d1e834084', 'request_id': 'req-f84a2e45-8957-42bc-b2f4-1ea1db339285'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:53.947523 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f84a2e45-8957-42bc-b2f4-1ea1db339285 admin admin] The token cache is being invalidate because project 10e0d1a2253742f5aed69a5d1e834084 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:28:53.947814 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f84a2e45-8957-42bc-b2f4-1ea1db339285 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:53.948451 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f84a2e45-8957-42bc-b2f4-1ea1db339285 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:53.949989 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1065/2129] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:28:53 2022] DELETE /identity/v3/projects/10e0d1a2253742f5aed69a5d1e834084 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:28:53.954070 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee014d-932f-4bdd-b13d-dd4d8f6e9c60 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:53.955520 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-39ee014d-932f-4bdd-b13d-dd4d8f6e9c60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:53.969199 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee014d-932f-4bdd-b13d-dd4d8f6e9c60 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:54.028292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0582b5de-48da-43b0-a062-7c4db57cc46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:54.028847 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0582b5de-48da-43b0-a062-7c4db57cc46d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:54.029154 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0582b5de-48da-43b0-a062-7c4db57cc46d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:54.029410 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0582b5de-48da-43b0-a062-7c4db57cc46d admin admin] PATH_INFO: `/v3/users/835de29e84ca4e269c9b412781b1c900` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:54.031299 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0582b5de-48da-43b0-a062-7c4db57cc46d admin admin] RBAC: Authorizing `identity:delete_user(user_id=835de29e84ca4e269c9b412781b1c900)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:54.033332 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0582b5de-48da-43b0-a062-7c4db57cc46d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:54.081044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0582b5de-48da-43b0-a062-7c4db57cc46d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '835de29e84ca4e269c9b412781b1c900'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:54.090126 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0582b5de-48da-43b0-a062-7c4db57cc46d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '835de29e84ca4e269c9b412781b1c900'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:54.090742 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0582b5de-48da-43b0-a062-7c4db57cc46d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '835de29e84ca4e269c9b412781b1c900'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:54.102921 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:28:54.104898 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0582b5de-48da-43b0-a062-7c4db57cc46d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '835de29e84ca4e269c9b412781b1c900'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:54.105751 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0582b5de-48da-43b0-a062-7c4db57cc46d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '835de29e84ca4e269c9b412781b1c900'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:54.120570 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1065/2130] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:28:53 2022] DELETE /identity/v3/users/835de29e84ca4e269c9b412781b1c900 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:28:54.408025 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84a2e45-8957-42bc-b2f4-1ea1db339285 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:54.410269 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f84a2e45-8957-42bc-b2f4-1ea1db339285 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:54.425880 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84a2e45-8957-42bc-b2f4-1ea1db339285 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:54.493490 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17acd17-ab5b-4450-9843-65e891ad5a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:54.494209 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17acd17-ab5b-4450-9843-65e891ad5a40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:54.494495 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17acd17-ab5b-4450-9843-65e891ad5a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:54.494771 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17acd17-ab5b-4450-9843-65e891ad5a40 admin admin] PATH_INFO: `/v3/projects/6b08d9c9534f495d9df4245c3fe7521c` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:54.495989 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17acd17-ab5b-4450-9843-65e891ad5a40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b08d9c9534f495d9df4245c3fe7521c)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:54.497941 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17acd17-ab5b-4450-9843-65e891ad5a40 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:54.569768 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c17acd17-ab5b-4450-9843-65e891ad5a40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b08d9c9534f495d9df4245c3fe7521c', 'request_id': 'req-c17acd17-ab5b-4450-9843-65e891ad5a40'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:54.579833 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c17acd17-ab5b-4450-9843-65e891ad5a40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b08d9c9534f495d9df4245c3fe7521c', 'request_id': 'req-c17acd17-ab5b-4450-9843-65e891ad5a40'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:54.585656 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c17acd17-ab5b-4450-9843-65e891ad5a40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b08d9c9534f495d9df4245c3fe7521c', 'request_id': 'req-c17acd17-ab5b-4450-9843-65e891ad5a40'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:54.587572 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:28:54.598226 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c17acd17-ab5b-4450-9843-65e891ad5a40 admin admin] The token cache is being invalidate because project 6b08d9c9534f495d9df4245c3fe7521c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:28:54.599283 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c17acd17-ab5b-4450-9843-65e891ad5a40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:54.599742 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c17acd17-ab5b-4450-9843-65e891ad5a40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:54.602242 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1066/2131] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:28:54 2022] DELETE /identity/v3/projects/6b08d9c9534f495d9df4245c3fe7521c => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:28:54.607066 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d505410-30ac-42ac-a58f-57fb390b7177 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:54.607572 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d505410-30ac-42ac-a58f-57fb390b7177 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:54.607792 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d505410-30ac-42ac-a58f-57fb390b7177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:54.627702 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-5d505410-30ac-42ac-a58f-57fb390b7177 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:28:54.628928 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5d505410-30ac-42ac-a58f-57fb390b7177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:54.691140 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1066/2132] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:28:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:28:54.699144 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e9bf41-58b9-4096-90d4-736b0ca763b6 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:54.699430 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e9bf41-58b9-4096-90d4-736b0ca763b6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:54.699710 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e9bf41-58b9-4096-90d4-736b0ca763b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:54.721477 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e1e9bf41-58b9-4096-90d4-736b0ca763b6 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:28:54.722705 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e1e9bf41-58b9-4096-90d4-736b0ca763b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:54.731553 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1067/2133] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:28:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:28:54.735962 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d505410-30ac-42ac-a58f-57fb390b7177 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:54.746118 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d505410-30ac-42ac-a58f-57fb390b7177 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:54.751149 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703725ff-98b9-4a62-9c4d-0a21d0746645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:54.751846 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703725ff-98b9-4a62-9c4d-0a21d0746645 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:54.752193 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703725ff-98b9-4a62-9c4d-0a21d0746645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:54.752542 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703725ff-98b9-4a62-9c4d-0a21d0746645 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:54.753328 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703725ff-98b9-4a62-9c4d-0a21d0746645 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:28:54.753699 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703725ff-98b9-4a62-9c4d-0a21d0746645 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:54.755492 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703725ff-98b9-4a62-9c4d-0a21d0746645 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:54.767756 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1067/2134] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:28:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:28:54.772177 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e9bf41-58b9-4096-90d4-736b0ca763b6 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:54.782114 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e9bf41-58b9-4096-90d4-736b0ca763b6 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:54.788385 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfed9022-c77e-4df1-82c1-f2b2a798e81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:54.789568 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed9022-c77e-4df1-82c1-f2b2a798e81a admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:54.789860 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed9022-c77e-4df1-82c1-f2b2a798e81a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:54.790125 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed9022-c77e-4df1-82c1-f2b2a798e81a admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:54.790780 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfed9022-c77e-4df1-82c1-f2b2a798e81a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:54.792721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfed9022-c77e-4df1-82c1-f2b2a798e81a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:54.825717 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1068/2135] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:28:54 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:28:54.829613 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703725ff-98b9-4a62-9c4d-0a21d0746645 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:54.842240 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703725ff-98b9-4a62-9c4d-0a21d0746645 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:54.908939 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb6e70-1495-450f-b64d-656dcc03f682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:54.909575 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb6e70-1495-450f-b64d-656dcc03f682 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:54.910351 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb6e70-1495-450f-b64d-656dcc03f682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:54.910351 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb6e70-1495-450f-b64d-656dcc03f682 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:54.910799 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eb6e70-1495-450f-b64d-656dcc03f682 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:54.913044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eb6e70-1495-450f-b64d-656dcc03f682 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:54.941189 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1068/2136] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:28:54 2022] POST /identity/v3/users => generated 286 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:28:54.942623 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfed9022-c77e-4df1-82c1-f2b2a798e81a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:54.953513 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfed9022-c77e-4df1-82c1-f2b2a798e81a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:54.958508 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25819596-c98d-401b-817a-9cd3d182dccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:54.959421 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25819596-c98d-401b-817a-9cd3d182dccb admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:54.959768 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25819596-c98d-401b-817a-9cd3d182dccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:54.960132 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25819596-c98d-401b-817a-9cd3d182dccb admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:54.960916 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25819596-c98d-401b-817a-9cd3d182dccb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:54.962624 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25819596-c98d-401b-817a-9cd3d182dccb admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:54.975447 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1069/2137] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:28:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:28:54.979770 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb6e70-1495-450f-b64d-656dcc03f682 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:54.991187 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb6e70-1495-450f-b64d-656dcc03f682 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:54.996206 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1369432e-d91d-4c4d-a629-7d76dfb7c1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:54.996840 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1369432e-d91d-4c4d-a629-7d76dfb7c1a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:54.997114 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1369432e-d91d-4c4d-a629-7d76dfb7c1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:54.997379 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1369432e-d91d-4c4d-a629-7d76dfb7c1a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:54.998083 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1369432e-d91d-4c4d-a629-7d76dfb7c1a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:54.999759 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1369432e-d91d-4c4d-a629-7d76dfb7c1a0 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:55.015201 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1069/2138] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:28:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:28:55.016385 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25819596-c98d-401b-817a-9cd3d182dccb admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:55.027216 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25819596-c98d-401b-817a-9cd3d182dccb admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:55.032370 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b3e0db-d2d1-49bb-b967-10956440159b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:55.033066 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b3e0db-d2d1-49bb-b967-10956440159b admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:55.033891 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b3e0db-d2d1-49bb-b967-10956440159b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:55.034194 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b3e0db-d2d1-49bb-b967-10956440159b admin admin] PATH_INFO: `/v3/projects/b02ebfd3364e44aa930e03353f494750/users/60a3557e19be4a9e90149ff0fdf974ff/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:55.051024 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b3e0db-d2d1-49bb-b967-10956440159b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b02ebfd3364e44aa930e03353f494750, user_id=60a3557e19be4a9e90149ff0fdf974ff, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:55.053756 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b3e0db-d2d1-49bb-b967-10956440159b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:55.070780 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1070/2139] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:28:55 2022] PUT /identity/v3/projects/b02ebfd3364e44aa930e03353f494750/users/60a3557e19be4a9e90149ff0fdf974ff/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:28:55.084022 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9eb037-6684-4f1d-a9db-b553d1bb0375 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:55.084863 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9eb037-6684-4f1d-a9db-b553d1bb0375 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:55.085301 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9eb037-6684-4f1d-a9db-b553d1bb0375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:55.122387 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-4c9eb037-6684-4f1d-a9db-b553d1bb0375 None None] MFA Rules not processed for user `60a3557e19be4a9e90149ff0fdf974ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:28:55.123618 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4c9eb037-6684-4f1d-a9db-b553d1bb0375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:55.172668 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:28:55.200900 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1070/2140] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:28:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:28:55.212811 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b3e0db-d2d1-49bb-b967-10956440159b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:55.214631 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-12b3e0db-d2d1-49bb-b967-10956440159b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:55.284538 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b3e0db-d2d1-49bb-b967-10956440159b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:55.289467 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe74d07-6290-437f-904f-cfa6d221ae31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:55.290127 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe74d07-6290-437f-904f-cfa6d221ae31 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:55.290416 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe74d07-6290-437f-904f-cfa6d221ae31 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:55.290698 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe74d07-6290-437f-904f-cfa6d221ae31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:55.302849 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe74d07-6290-437f-904f-cfa6d221ae31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:55.304412 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe74d07-6290-437f-904f-cfa6d221ae31 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:55.315749 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1071/2141] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:28:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:28:55.344982 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9eb037-6684-4f1d-a9db-b553d1bb0375 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:55.442400 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9eb037-6684-4f1d-a9db-b553d1bb0375 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:55.453332 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66fbfcf-5adf-48a4-a908-d9926655a939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:55.454391 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66fbfcf-5adf-48a4-a908-d9926655a939 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:55.454391 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66fbfcf-5adf-48a4-a908-d9926655a939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:55.454619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66fbfcf-5adf-48a4-a908-d9926655a939 admin admin] PATH_INFO: `/v3/users/60a3557e19be4a9e90149ff0fdf974ff` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:55.459086 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66fbfcf-5adf-48a4-a908-d9926655a939 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60a3557e19be4a9e90149ff0fdf974ff)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:55.467813 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66fbfcf-5adf-48a4-a908-d9926655a939 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:55.563089 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c66fbfcf-5adf-48a4-a908-d9926655a939 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60a3557e19be4a9e90149ff0fdf974ff'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:55.598889 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c66fbfcf-5adf-48a4-a908-d9926655a939 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60a3557e19be4a9e90149ff0fdf974ff'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:55.599655 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c66fbfcf-5adf-48a4-a908-d9926655a939 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60a3557e19be4a9e90149ff0fdf974ff'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:55.620897 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c66fbfcf-5adf-48a4-a908-d9926655a939 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60a3557e19be4a9e90149ff0fdf974ff'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:55.622647 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c66fbfcf-5adf-48a4-a908-d9926655a939 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60a3557e19be4a9e90149ff0fdf974ff'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:55.656490 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1071/2142] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:28:55 2022] DELETE /identity/v3/users/60a3557e19be4a9e90149ff0fdf974ff => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:28:55.661489 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061fa537-1d69-4d81-9a54-b658a8dc0b6a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:55.661780 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061fa537-1d69-4d81-9a54-b658a8dc0b6a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:55.662058 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061fa537-1d69-4d81-9a54-b658a8dc0b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:55.688651 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-061fa537-1d69-4d81-9a54-b658a8dc0b6a None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:28:55.691421 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-061fa537-1d69-4d81-9a54-b658a8dc0b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:55.809125 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1072/2143] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:28:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:28:55.824676 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66fbfcf-5adf-48a4-a908-d9926655a939 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:55.827110 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c66fbfcf-5adf-48a4-a908-d9926655a939 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:55.935482 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66fbfcf-5adf-48a4-a908-d9926655a939 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:55.941897 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570025c1-a9a9-4bae-948d-3dd569fee018 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:55.943103 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570025c1-a9a9-4bae-948d-3dd569fee018 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:55.943613 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570025c1-a9a9-4bae-948d-3dd569fee018 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:55.944162 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570025c1-a9a9-4bae-948d-3dd569fee018 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:55.958334 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570025c1-a9a9-4bae-948d-3dd569fee018 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:55.960675 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570025c1-a9a9-4bae-948d-3dd569fee018 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:55.989525 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1072/2144] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:28:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:28:56.996129 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061fa537-1d69-4d81-9a54-b658a8dc0b6a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:56.997766 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-061fa537-1d69-4d81-9a54-b658a8dc0b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:57.011289 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061fa537-1d69-4d81-9a54-b658a8dc0b6a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:57.016840 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7667edab-8a64-446b-ae8d-c11816a9db83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:57.017571 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7667edab-8a64-446b-ae8d-c11816a9db83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:57.018054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7667edab-8a64-446b-ae8d-c11816a9db83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:57.018653 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7667edab-8a64-446b-ae8d-c11816a9db83 admin admin] PATH_INFO: `/v3/projects/b02ebfd3364e44aa930e03353f494750` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:57.021226 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7667edab-8a64-446b-ae8d-c11816a9db83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b02ebfd3364e44aa930e03353f494750)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:57.023304 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7667edab-8a64-446b-ae8d-c11816a9db83 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:57.069986 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:28:57.105183 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7667edab-8a64-446b-ae8d-c11816a9db83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b02ebfd3364e44aa930e03353f494750', 'request_id': 'req-7667edab-8a64-446b-ae8d-c11816a9db83'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:57.119934 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7667edab-8a64-446b-ae8d-c11816a9db83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b02ebfd3364e44aa930e03353f494750', 'request_id': 'req-7667edab-8a64-446b-ae8d-c11816a9db83'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:57.136534 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7667edab-8a64-446b-ae8d-c11816a9db83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b02ebfd3364e44aa930e03353f494750', 'request_id': 'req-7667edab-8a64-446b-ae8d-c11816a9db83'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:57.150655 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7667edab-8a64-446b-ae8d-c11816a9db83 admin admin] The token cache is being invalidate because project b02ebfd3364e44aa930e03353f494750 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:28:57.162695 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7667edab-8a64-446b-ae8d-c11816a9db83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:57.163511 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7667edab-8a64-446b-ae8d-c11816a9db83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:28:57.165122 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1073/2145] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:28:56 2022] DELETE /identity/v3/projects/b02ebfd3364e44aa930e03353f494750 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:28:57.170165 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363360d4-015e-4e61-bf02-a0f2517a4e07 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:57.170644 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363360d4-015e-4e61-bf02-a0f2517a4e07 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:57.170743 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363360d4-015e-4e61-bf02-a0f2517a4e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:57.180399 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:28:57.196631 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-363360d4-015e-4e61-bf02-a0f2517a4e07 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:28:57.198984 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-363360d4-015e-4e61-bf02-a0f2517a4e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:57.299107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1073/2146] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:28:57.306511 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed3a56-4f56-4d79-bae1-9f25ed732b8d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:57.307062 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed3a56-4f56-4d79-bae1-9f25ed732b8d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:57.307495 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed3a56-4f56-4d79-bae1-9f25ed732b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:57.337746 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-92ed3a56-4f56-4d79-bae1-9f25ed732b8d None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:28:57.339124 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-92ed3a56-4f56-4d79-bae1-9f25ed732b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:57.347665 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1074/2147] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:28:57.352684 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363360d4-015e-4e61-bf02-a0f2517a4e07 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:57.363841 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363360d4-015e-4e61-bf02-a0f2517a4e07 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:57.377603 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f9a8cc-3c80-43f3-8679-aeb6ae1f0ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:57.378496 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9a8cc-3c80-43f3-8679-aeb6ae1f0ed6 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:57.378948 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9a8cc-3c80-43f3-8679-aeb6ae1f0ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:57.379383 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9a8cc-3c80-43f3-8679-aeb6ae1f0ed6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:57.380438 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f9a8cc-3c80-43f3-8679-aeb6ae1f0ed6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:28:57.381063 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f9a8cc-3c80-43f3-8679-aeb6ae1f0ed6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:57.385705 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f9a8cc-3c80-43f3-8679-aeb6ae1f0ed6 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:57.408382 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed3a56-4f56-4d79-bae1-9f25ed732b8d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:57.409512 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1074/2148] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:28:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:28:57.420891 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed3a56-4f56-4d79-bae1-9f25ed732b8d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:57.431516 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf96c144-d634-43ea-b768-9db1675e1bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:57.432411 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96c144-d634-43ea-b768-9db1675e1bbd admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:57.432931 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96c144-d634-43ea-b768-9db1675e1bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:57.433561 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96c144-d634-43ea-b768-9db1675e1bbd admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:57.435093 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf96c144-d634-43ea-b768-9db1675e1bbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:57.438945 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf96c144-d634-43ea-b768-9db1675e1bbd admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:57.476138 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1075/2149] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:28:57 2022] POST /identity/v3/projects => generated 371 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:28:57.481259 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f9a8cc-3c80-43f3-8679-aeb6ae1f0ed6 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:57.493041 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f9a8cc-3c80-43f3-8679-aeb6ae1f0ed6 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:57.561195 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7794ea6-ad1a-482f-8c05-5296bfed5e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:57.562166 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7794ea6-ad1a-482f-8c05-5296bfed5e07 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:57.562660 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7794ea6-ad1a-482f-8c05-5296bfed5e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:57.563088 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7794ea6-ad1a-482f-8c05-5296bfed5e07 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:57.563766 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7794ea6-ad1a-482f-8c05-5296bfed5e07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:57.566003 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7794ea6-ad1a-482f-8c05-5296bfed5e07 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:57.591446 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1075/2150] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:28:57 2022] POST /identity/v3/users => generated 286 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:28:57.595601 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf96c144-d634-43ea-b768-9db1675e1bbd admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:57.606051 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf96c144-d634-43ea-b768-9db1675e1bbd admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:57.610815 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575ba95-3ebb-4f6d-b40b-1a3dd4846d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:57.611498 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575ba95-3ebb-4f6d-b40b-1a3dd4846d72 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:57.611774 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575ba95-3ebb-4f6d-b40b-1a3dd4846d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:57.612045 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575ba95-3ebb-4f6d-b40b-1a3dd4846d72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:57.612755 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6575ba95-3ebb-4f6d-b40b-1a3dd4846d72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:57.614546 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6575ba95-3ebb-4f6d-b40b-1a3dd4846d72 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:57.637653 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1076/2151] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:28:57.641751 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7794ea6-ad1a-482f-8c05-5296bfed5e07 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:57.652430 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7794ea6-ad1a-482f-8c05-5296bfed5e07 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:57.658214 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b4b96-7e19-4f47-9599-e64d9274ca5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:57.659258 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b4b96-7e19-4f47-9599-e64d9274ca5b admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:57.659538 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b4b96-7e19-4f47-9599-e64d9274ca5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:57.659858 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b4b96-7e19-4f47-9599-e64d9274ca5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:57.660572 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72b4b96-7e19-4f47-9599-e64d9274ca5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:57.662541 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72b4b96-7e19-4f47-9599-e64d9274ca5b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:57.675930 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1076/2152] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:28:57.680330 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575ba95-3ebb-4f6d-b40b-1a3dd4846d72 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:57.690443 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575ba95-3ebb-4f6d-b40b-1a3dd4846d72 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:57.695460 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8e19e2-d991-4680-af2b-b04d994fd2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:57.696144 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e19e2-d991-4680-af2b-b04d994fd2e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:57.696450 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e19e2-d991-4680-af2b-b04d994fd2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:57.696719 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e19e2-d991-4680-af2b-b04d994fd2e6 admin admin] PATH_INFO: `/v3/projects/4183067b2b9c429d85238263419ec3f7/users/a4b9bf1486c745f0bd204307b942809a/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:57.711860 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8e19e2-d991-4680-af2b-b04d994fd2e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4183067b2b9c429d85238263419ec3f7, user_id=a4b9bf1486c745f0bd204307b942809a, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:57.714445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8e19e2-d991-4680-af2b-b04d994fd2e6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:57.730879 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1077/2153] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:28:57 2022] PUT /identity/v3/projects/4183067b2b9c429d85238263419ec3f7/users/a4b9bf1486c745f0bd204307b942809a/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:28:57.736397 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aaf9bf-39c5-4381-a407-da0a0054e7c2 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:57.736812 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aaf9bf-39c5-4381-a407-da0a0054e7c2 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:57.737532 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aaf9bf-39c5-4381-a407-da0a0054e7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:57.759402 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-62aaf9bf-39c5-4381-a407-da0a0054e7c2 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:28:57.760833 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-62aaf9bf-39c5-4381-a407-da0a0054e7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:57.866808 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1077/2154] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:28:57.880316 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8e19e2-d991-4680-af2b-b04d994fd2e6 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:28:57.883659 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ab8e19e2-d991-4680-af2b-b04d994fd2e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:28:57.972420 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8e19e2-d991-4680-af2b-b04d994fd2e6 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:28:57.978940 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2537aec8-c88d-49a8-a03b-1141c124ee14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:28:57.979567 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537aec8-c88d-49a8-a03b-1141c124ee14 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:28:57.979848 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537aec8-c88d-49a8-a03b-1141c124ee14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:28:57.980122 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537aec8-c88d-49a8-a03b-1141c124ee14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:28:57.989882 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2537aec8-c88d-49a8-a03b-1141c124ee14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:28:57.991402 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2537aec8-c88d-49a8-a03b-1141c124ee14 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:28:58.001962 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1078/2155] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:28:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:29:00.740657 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d754f36-fdd3-43ae-ad43-f3793e2f0b89 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:29:00.741133 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d754f36-fdd3-43ae-ad43-f3793e2f0b89 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:29:00.741213 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d754f36-fdd3-43ae-ad43-f3793e2f0b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:29:00.767889 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-0d754f36-fdd3-43ae-ad43-f3793e2f0b89 None None] MFA Rules not processed for user `a3b4d90bcfc34d2d9ba1c40316e09858`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:29:00.769267 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0d754f36-fdd3-43ae-ad43-f3793e2f0b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:29:00.780413 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:29:00.858179 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1078/2156] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:29:00 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:29:03.754871 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb060537-258f-4e14-b4fa-adebed7f78d6 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:29:03.755423 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb060537-258f-4e14-b4fa-adebed7f78d6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:29:03.755600 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb060537-258f-4e14-b4fa-adebed7f78d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:29:03.796976 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-fb060537-258f-4e14-b4fa-adebed7f78d6 None None] MFA Rules not processed for user `a4b9bf1486c745f0bd204307b942809a`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:29:03.798512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fb060537-258f-4e14-b4fa-adebed7f78d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:29:03.861966 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:29:03.896216 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1079/2157] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:29:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:29:03.897261 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d754f36-fdd3-43ae-ad43-f3793e2f0b89 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:29:03.910133 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d754f36-fdd3-43ae-ad43-f3793e2f0b89 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:29:03.915360 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6807c57-cee5-4101-b8b9-087f97973eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:29:03.916260 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6807c57-cee5-4101-b8b9-087f97973eef admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:29:03.916916 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6807c57-cee5-4101-b8b9-087f97973eef admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:29:03.917349 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6807c57-cee5-4101-b8b9-087f97973eef admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:29:03.918269 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6807c57-cee5-4101-b8b9-087f97973eef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:29:03.920775 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6807c57-cee5-4101-b8b9-087f97973eef admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:29:03.974317 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb060537-258f-4e14-b4fa-adebed7f78d6 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:29:03.980753 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1079/2158] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:29:03 2022] POST /identity/v3/projects => generated 373 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:29:03.988890 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb060537-258f-4e14-b4fa-adebed7f78d6 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:29:04.127709 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1118d1-8b45-4495-b790-8a4cf5973191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:29:04.128453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1118d1-8b45-4495-b790-8a4cf5973191 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:29:04.128802 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1118d1-8b45-4495-b790-8a4cf5973191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:29:04.129143 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1118d1-8b45-4495-b790-8a4cf5973191 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:29:04.129952 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1118d1-8b45-4495-b790-8a4cf5973191 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:29:04.133711 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1118d1-8b45-4495-b790-8a4cf5973191 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:29:04.193211 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1080/2159] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:29:03 2022] POST /identity/v3/users => generated 293 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:29:04.197184 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6807c57-cee5-4101-b8b9-087f97973eef admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:29:04.209771 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6807c57-cee5-4101-b8b9-087f97973eef admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:29:04.216564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b45530-e8b1-42db-bba0-a34d3094d9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:29:04.217567 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b45530-e8b1-42db-bba0-a34d3094d9ca admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:29:04.218272 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b45530-e8b1-42db-bba0-a34d3094d9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:29:04.218702 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b45530-e8b1-42db-bba0-a34d3094d9ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:29:04.219610 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b45530-e8b1-42db-bba0-a34d3094d9ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:29:04.221673 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b45530-e8b1-42db-bba0-a34d3094d9ca admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:29:04.235249 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1080/2160] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:29:04.245092 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1118d1-8b45-4495-b790-8a4cf5973191 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:29:04.258728 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1118d1-8b45-4495-b790-8a4cf5973191 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:29:04.264228 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6992db-05d6-417d-82a8-8c6c39b1f2c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:29:04.265181 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6992db-05d6-417d-82a8-8c6c39b1f2c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:29:04.265353 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6992db-05d6-417d-82a8-8c6c39b1f2c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:29:04.265705 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6992db-05d6-417d-82a8-8c6c39b1f2c1 admin admin] PATH_INFO: `/v3/projects/bfc491299375406ab19f0723bef93f14/users/d155329c404d4e1ba27094e57210b8d7/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:29:04.288861 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6992db-05d6-417d-82a8-8c6c39b1f2c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfc491299375406ab19f0723bef93f14, user_id=d155329c404d4e1ba27094e57210b8d7, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:29:04.293327 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6992db-05d6-417d-82a8-8c6c39b1f2c1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:29:04.309366 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1081/2161] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:29:04 2022] PUT /identity/v3/projects/bfc491299375406ab19f0723bef93f14/users/d155329c404d4e1ba27094e57210b8d7/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:29:04.313300 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b45530-e8b1-42db-bba0-a34d3094d9ca admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:29:04.331689 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b45530-e8b1-42db-bba0-a34d3094d9ca admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:29:04.395121 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f615a68-0317-4b9b-8607-1f6d157337b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:29:04.395865 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f615a68-0317-4b9b-8607-1f6d157337b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:29:04.396332 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f615a68-0317-4b9b-8607-1f6d157337b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:29:04.396674 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f615a68-0317-4b9b-8607-1f6d157337b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:29:04.397472 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f615a68-0317-4b9b-8607-1f6d157337b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:29:04.399180 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f615a68-0317-4b9b-8607-1f6d157337b5 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:29:04.411733 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1081/2162] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:29:04.416095 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6992db-05d6-417d-82a8-8c6c39b1f2c1 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:29:04.427108 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6992db-05d6-417d-82a8-8c6c39b1f2c1 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:29:04.432131 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7184fb-2d6b-4aa3-b2bf-04fce5b3a4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:29:04.432806 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7184fb-2d6b-4aa3-b2bf-04fce5b3a4d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:29:04.433089 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7184fb-2d6b-4aa3-b2bf-04fce5b3a4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:29:04.433367 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7184fb-2d6b-4aa3-b2bf-04fce5b3a4d1 admin admin] PATH_INFO: `/v3/projects/bfc491299375406ab19f0723bef93f14/users/d155329c404d4e1ba27094e57210b8d7/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:29:04.435447 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7184fb-2d6b-4aa3-b2bf-04fce5b3a4d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfc491299375406ab19f0723bef93f14, user_id=d155329c404d4e1ba27094e57210b8d7, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:29:04.438149 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7184fb-2d6b-4aa3-b2bf-04fce5b3a4d1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:29:04.450188 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1082/2163] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:29:04 2022] PUT /identity/v3/projects/bfc491299375406ab19f0723bef93f14/users/d155329c404d4e1ba27094e57210b8d7/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:29:09.615443 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079b282-921e-470a-a7a7-b98e28955abd None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:29:09.615989 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079b282-921e-470a-a7a7-b98e28955abd None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:29:09.615989 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079b282-921e-470a-a7a7-b98e28955abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:29:09.646006 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-7079b282-921e-470a-a7a7-b98e28955abd None None] MFA Rules not processed for user `d155329c404d4e1ba27094e57210b8d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:29:09.647278 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7079b282-921e-470a-a7a7-b98e28955abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:29:09.726556 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1082/2164] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:29:09 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:29:09.741456 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7184fb-2d6b-4aa3-b2bf-04fce5b3a4d1 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:29:09.743247 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3c7184fb-2d6b-4aa3-b2bf-04fce5b3a4d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:29:09.832658 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7184fb-2d6b-4aa3-b2bf-04fce5b3a4d1 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:29:09.842513 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f459be7a-73dc-4fd4-bdae-027da491cccb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:29:09.843197 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f459be7a-73dc-4fd4-bdae-027da491cccb service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:29:09.843479 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f459be7a-73dc-4fd4-bdae-027da491cccb service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:29:09.843778 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f459be7a-73dc-4fd4-bdae-027da491cccb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:29:09.880393 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:29:09.905323 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f459be7a-73dc-4fd4-bdae-027da491cccb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:29:09.907718 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f459be7a-73dc-4fd4-bdae-027da491cccb service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:29:09.933907 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1083/2165] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:29:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:29:11.114520 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9e2bc-58cd-4243-b450-f808b192e19f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:29:11.115117 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9e2bc-58cd-4243-b450-f808b192e19f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:29:11.115117 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9e2bc-58cd-4243-b450-f808b192e19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:29:11.123160 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:29:11.137777 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-1fb9e2bc-58cd-4243-b450-f808b192e19f None None] MFA Rules not processed for user `a4b9bf1486c745f0bd204307b942809a`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:29:11.139030 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1fb9e2bc-58cd-4243-b450-f808b192e19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:29:11.146367 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1083/2166] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:29:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:29:19.914235 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=11 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:29:21.132752 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:29:25.472487 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71022ad9-dcf0-4f7d-94de-5fa3650bb872 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:29:25.473024 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71022ad9-dcf0-4f7d-94de-5fa3650bb872 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:29:25.473024 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71022ad9-dcf0-4f7d-94de-5fa3650bb872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:29:25.492914 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-71022ad9-dcf0-4f7d-94de-5fa3650bb872 None None] MFA Rules not processed for user `52fc3ba90dc045c2a5b7bb730da96acc`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:29:25.494551 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-71022ad9-dcf0-4f7d-94de-5fa3650bb872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:29:25.557668 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1084/2167] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 5 10:29:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:29:25.601875 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb9e2bc-58cd-4243-b450-f808b192e19f None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:29:25.603548 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1fb9e2bc-58cd-4243-b450-f808b192e19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:29:25.685556 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb9e2bc-58cd-4243-b450-f808b192e19f None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:29:25.690975 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ea2088-25c2-4bb6-bdcb-1e729751b12c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c80cf0e429f04f229aea1aa9293a93b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:29:25.691613 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea2088-25c2-4bb6-bdcb-1e729751b12c service swift] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:29:25.691895 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea2088-25c2-4bb6-bdcb-1e729751b12c service swift] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:29:25.692197 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea2088-25c2-4bb6-bdcb-1e729751b12c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:29:25.701394 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ea2088-25c2-4bb6-bdcb-1e729751b12c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:29:25.703042 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ea2088-25c2-4bb6-bdcb-1e729751b12c service swift] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:29:25.711604 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1084/2168] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 5 10:29:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 10:29:35.549101 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=25 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:29:35.708624 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=32 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:29:42.133589 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71022ad9-dcf0-4f7d-94de-5fa3650bb872 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:29:42.135306 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-71022ad9-dcf0-4f7d-94de-5fa3650bb872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:29:42.224209 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71022ad9-dcf0-4f7d-94de-5fa3650bb872 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:29:42.230771 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c43848-d029-4c03-914e-620885c2d290 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8743f5bb0be54d57a957617636b0b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:29:42.231436 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c43848-d029-4c03-914e-620885c2d290 service placement] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:29:42.231713 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c43848-d029-4c03-914e-620885c2d290 service placement] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:29:42.231986 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c43848-d029-4c03-914e-620885c2d290 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:29:42.234217 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-70c43848-d029-4c03-914e-620885c2d290 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:29:42.246927 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c43848-d029-4c03-914e-620885c2d290 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:29:42.248695 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c43848-d029-4c03-914e-620885c2d290 service placement] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:29:42.259258 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1085/2169] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:29:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:29:42.352541 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80ab21-2f1f-4f44-8de4-a1961c4b6ae9 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:29:42.353101 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80ab21-2f1f-4f44-8de4-a1961c4b6ae9 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:29:42.353201 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80ab21-2f1f-4f44-8de4-a1961c4b6ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:29:42.374583 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ad80ab21-2f1f-4f44-8de4-a1961c4b6ae9 None None] MFA Rules not processed for user `52fc3ba90dc045c2a5b7bb730da96acc`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:29:42.375808 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ad80ab21-2f1f-4f44-8de4-a1961c4b6ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:29:42.384089 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1085/2170] 10.222.0.40 () {58 vars in 967 bytes} [Fri Aug 5 10:29:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:29:42.399025 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c43848-d029-4c03-914e-620885c2d290 service placement] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:29:42.413021 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c43848-d029-4c03-914e-620885c2d290 service placement] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:29:42.419874 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4f6f68-2cee-43d9-8ce3-d10670e61932 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c80cf0e429f04f229aea1aa9293a93b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:29:42.420830 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f6f68-2cee-43d9-8ce3-d10670e61932 service swift] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:29:42.421225 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f6f68-2cee-43d9-8ce3-d10670e61932 service swift] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:29:42.421631 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f6f68-2cee-43d9-8ce3-d10670e61932 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:29:42.434756 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4f6f68-2cee-43d9-8ce3-d10670e61932 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:29:42.437507 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4f6f68-2cee-43d9-8ce3-d10670e61932 service swift] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:29:42.446830 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1086/2171] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 5 10:29:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 10:29:44.564000 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad80ab21-2f1f-4f44-8de4-a1961c4b6ae9 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:29:44.565610 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ad80ab21-2f1f-4f44-8de4-a1961c4b6ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:29:44.648356 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad80ab21-2f1f-4f44-8de4-a1961c4b6ae9 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:29:44.653648 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df3ba3-d940-44ab-b044-dd4bc657bc86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:29:44.654236 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df3ba3-d940-44ab-b044-dd4bc657bc86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:29:44.654514 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df3ba3-d940-44ab-b044-dd4bc657bc86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:29:44.654832 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df3ba3-d940-44ab-b044-dd4bc657bc86 admin admin] PATH_INFO: `/v3/users/7723cdbd378743378b1fccc2d15bc53b` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:29:44.656264 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2df3ba3-d940-44ab-b044-dd4bc657bc86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7723cdbd378743378b1fccc2d15bc53b)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:29:44.658534 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2df3ba3-d940-44ab-b044-dd4bc657bc86 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:29:44.710231 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d2df3ba3-d940-44ab-b044-dd4bc657bc86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7723cdbd378743378b1fccc2d15bc53b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:29:44.719536 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d2df3ba3-d940-44ab-b044-dd4bc657bc86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7723cdbd378743378b1fccc2d15bc53b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:29:44.720168 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d2df3ba3-d940-44ab-b044-dd4bc657bc86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7723cdbd378743378b1fccc2d15bc53b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:29:44.732850 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d2df3ba3-d940-44ab-b044-dd4bc657bc86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7723cdbd378743378b1fccc2d15bc53b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:29:44.733630 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d2df3ba3-d940-44ab-b044-dd4bc657bc86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7723cdbd378743378b1fccc2d15bc53b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:29:44.746620 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1086/2172] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:29:44 2022] DELETE /identity/v3/users/7723cdbd378743378b1fccc2d15bc53b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:29:44.944030 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4f6f68-2cee-43d9-8ce3-d10670e61932 service swift] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:29:44.956724 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4f6f68-2cee-43d9-8ce3-d10670e61932 service swift] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:29:45.026069 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8d902a-53ea-411f-9320-b0074216f61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:29:45.026842 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8d902a-53ea-411f-9320-b0074216f61d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:29:45.027169 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8d902a-53ea-411f-9320-b0074216f61d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:29:45.027449 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8d902a-53ea-411f-9320-b0074216f61d admin admin] PATH_INFO: `/v3/projects/a4a8139ee97548e89fcb7b38c5bd99fe` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:29:45.028636 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8d902a-53ea-411f-9320-b0074216f61d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4a8139ee97548e89fcb7b38c5bd99fe)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:29:45.030569 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8d902a-53ea-411f-9320-b0074216f61d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:29:45.084451 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ca8d902a-53ea-411f-9320-b0074216f61d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4a8139ee97548e89fcb7b38c5bd99fe', 'request_id': 'req-ca8d902a-53ea-411f-9320-b0074216f61d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:29:45.101539 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ca8d902a-53ea-411f-9320-b0074216f61d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4a8139ee97548e89fcb7b38c5bd99fe', 'request_id': 'req-ca8d902a-53ea-411f-9320-b0074216f61d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:29:45.108148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ca8d902a-53ea-411f-9320-b0074216f61d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4a8139ee97548e89fcb7b38c5bd99fe', 'request_id': 'req-ca8d902a-53ea-411f-9320-b0074216f61d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:29:45.119674 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ca8d902a-53ea-411f-9320-b0074216f61d admin admin] The token cache is being invalidate because project a4a8139ee97548e89fcb7b38c5bd99fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:29:45.119963 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ca8d902a-53ea-411f-9320-b0074216f61d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:29:45.120660 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ca8d902a-53ea-411f-9320-b0074216f61d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:29:45.122245 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1087/2173] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:29:44 2022] DELETE /identity/v3/projects/a4a8139ee97548e89fcb7b38c5bd99fe => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:29:45.128710 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c5fab2-d938-46b4-a39e-bd079047cf53 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:29:45.129025 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c5fab2-d938-46b4-a39e-bd079047cf53 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:29:45.129305 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c5fab2-d938-46b4-a39e-bd079047cf53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:29:45.149286 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-d4c5fab2-d938-46b4-a39e-bd079047cf53 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:29:45.151523 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d4c5fab2-d938-46b4-a39e-bd079047cf53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:29:45.238170 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1087/2174] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:29:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:29:45.240040 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee961b-a56d-430e-956e-c5a23a3b3d70 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:29:45.240326 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee961b-a56d-430e-956e-c5a23a3b3d70 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:29:45.240602 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee961b-a56d-430e-956e-c5a23a3b3d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:29:45.255226 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:29:45.260789 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-7dee961b-a56d-430e-956e-c5a23a3b3d70 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:29:45.262846 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7dee961b-a56d-430e-956e-c5a23a3b3d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:29:45.271167 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1088/2175] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:29:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:29:45.275524 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c5fab2-d938-46b4-a39e-bd079047cf53 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:29:45.285914 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c5fab2-d938-46b4-a39e-bd079047cf53 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:29:45.290836 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee72d2-34a9-4bf0-adf1-62dc6316bd24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:29:45.291533 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee72d2-34a9-4bf0-adf1-62dc6316bd24 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:29:45.291814 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee72d2-34a9-4bf0-adf1-62dc6316bd24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:29:45.292078 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee72d2-34a9-4bf0-adf1-62dc6316bd24 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:29:45.292853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee72d2-34a9-4bf0-adf1-62dc6316bd24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:29:45.293182 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee72d2-34a9-4bf0-adf1-62dc6316bd24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:29:45.294874 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee72d2-34a9-4bf0-adf1-62dc6316bd24 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:29:45.304023 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:29:45.307732 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1088/2176] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:29:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:29:45.312352 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dee961b-a56d-430e-956e-c5a23a3b3d70 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:29:45.322053 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dee961b-a56d-430e-956e-c5a23a3b3d70 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:29:45.327590 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782145d8-2f95-4c15-bb0f-8a72592afdef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:29:45.328225 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782145d8-2f95-4c15-bb0f-8a72592afdef admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:29:45.328505 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782145d8-2f95-4c15-bb0f-8a72592afdef admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:29:45.328827 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782145d8-2f95-4c15-bb0f-8a72592afdef admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:29:45.330081 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782145d8-2f95-4c15-bb0f-8a72592afdef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:29:45.332068 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782145d8-2f95-4c15-bb0f-8a72592afdef admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:29:45.362951 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1089/2177] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:29:45 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:29:45.366307 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee72d2-34a9-4bf0-adf1-62dc6316bd24 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:29:45.377282 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee72d2-34a9-4bf0-adf1-62dc6316bd24 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:29:45.439296 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb83cc7-54d1-4722-bd52-71a218723211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:29:45.439835 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb83cc7-54d1-4722-bd52-71a218723211 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:29:45.440117 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb83cc7-54d1-4722-bd52-71a218723211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:29:45.440389 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb83cc7-54d1-4722-bd52-71a218723211 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:29:45.441056 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb83cc7-54d1-4722-bd52-71a218723211 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:29:45.442995 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb83cc7-54d1-4722-bd52-71a218723211 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:29:45.467230 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1089/2178] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:29:45 2022] POST /identity/v3/users => generated 286 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:29:45.471428 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782145d8-2f95-4c15-bb0f-8a72592afdef admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:29:45.482081 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782145d8-2f95-4c15-bb0f-8a72592afdef admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:29:45.488675 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc07de48-21c0-48ef-960d-fac5459665ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:29:45.489580 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07de48-21c0-48ef-960d-fac5459665ca admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:29:45.489992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07de48-21c0-48ef-960d-fac5459665ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:29:45.490361 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07de48-21c0-48ef-960d-fac5459665ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:29:45.491484 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc07de48-21c0-48ef-960d-fac5459665ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:29:45.494037 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc07de48-21c0-48ef-960d-fac5459665ca admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:29:45.509649 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1090/2179] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:29:45.513812 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb83cc7-54d1-4722-bd52-71a218723211 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:29:45.524667 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb83cc7-54d1-4722-bd52-71a218723211 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:29:45.529844 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69189006-438d-47a0-93ed-f24d0b267d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:29:45.530459 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69189006-438d-47a0-93ed-f24d0b267d9d admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:29:45.530742 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69189006-438d-47a0-93ed-f24d0b267d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:29:45.531084 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69189006-438d-47a0-93ed-f24d0b267d9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:29:45.531835 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69189006-438d-47a0-93ed-f24d0b267d9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:29:45.533761 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69189006-438d-47a0-93ed-f24d0b267d9d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:29:45.545026 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1090/2180] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:29:45.549793 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc07de48-21c0-48ef-960d-fac5459665ca admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:29:45.563282 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc07de48-21c0-48ef-960d-fac5459665ca admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:29:45.572606 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be904ce9-c09b-4d95-bdc8-4e8d2e9c7e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:29:45.573607 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be904ce9-c09b-4d95-bdc8-4e8d2e9c7e32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:29:45.573994 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be904ce9-c09b-4d95-bdc8-4e8d2e9c7e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:29:45.574267 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be904ce9-c09b-4d95-bdc8-4e8d2e9c7e32 admin admin] PATH_INFO: `/v3/projects/3461d1c72706452b85c97019b722bcf8/users/23c584397b454762a7001e6eb88203ca/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:29:45.590444 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be904ce9-c09b-4d95-bdc8-4e8d2e9c7e32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3461d1c72706452b85c97019b722bcf8, user_id=23c584397b454762a7001e6eb88203ca, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:29:45.593123 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be904ce9-c09b-4d95-bdc8-4e8d2e9c7e32 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:29:45.608870 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1091/2181] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:29:45 2022] PUT /identity/v3/projects/3461d1c72706452b85c97019b722bcf8/users/23c584397b454762a7001e6eb88203ca/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:29:45.621412 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b967dbc-ec6a-4401-beed-074d8642a349 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:29:45.621952 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b967dbc-ec6a-4401-beed-074d8642a349 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:29:45.621952 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b967dbc-ec6a-4401-beed-074d8642a349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:29:45.649968 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-3b967dbc-ec6a-4401-beed-074d8642a349 None None] MFA Rules not processed for user `23c584397b454762a7001e6eb88203ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:29:45.651193 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3b967dbc-ec6a-4401-beed-074d8642a349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:29:45.719317 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1091/2182] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:29:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:29:45.733201 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be904ce9-c09b-4d95-bdc8-4e8d2e9c7e32 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:29:45.735352 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-be904ce9-c09b-4d95-bdc8-4e8d2e9c7e32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:29:45.795124 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be904ce9-c09b-4d95-bdc8-4e8d2e9c7e32 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:29:45.798848 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44e506-647b-4aab-bcfe-17c2b1dc03b1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac0a8cf7f8b74e88a04444ec38547528', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:29:45.799497 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44e506-647b-4aab-bcfe-17c2b1dc03b1 service glance] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:29:45.799777 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44e506-647b-4aab-bcfe-17c2b1dc03b1 service glance] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:29:45.800050 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44e506-647b-4aab-bcfe-17c2b1dc03b1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:29:45.808718 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da44e506-647b-4aab-bcfe-17c2b1dc03b1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:29:45.810161 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da44e506-647b-4aab-bcfe-17c2b1dc03b1 service glance] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:29:45.819273 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1092/2183] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:29:45 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:29:55.716060 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=66,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:29:55.815065 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=50,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:30:03.576422 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f40ab8-93e1-4018-9c73-eb276d926c9a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:30:03.576925 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f40ab8-93e1-4018-9c73-eb276d926c9a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:30:03.577020 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f40ab8-93e1-4018-9c73-eb276d926c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:30:03.598402 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-36f40ab8-93e1-4018-9c73-eb276d926c9a None None] MFA Rules not processed for user `706dbfa719c04fd0a3704ebd7eb812cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:30:03.599897 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-36f40ab8-93e1-4018-9c73-eb276d926c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:30:03.657254 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1092/2184] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:30:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:30:04.741370 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44e506-647b-4aab-bcfe-17c2b1dc03b1 service glance] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:30:04.743118 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-da44e506-647b-4aab-bcfe-17c2b1dc03b1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:30:04.804694 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44e506-647b-4aab-bcfe-17c2b1dc03b1 service glance] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:30:04.808325 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7349e0a4-114e-4fb5-a6d9-603466615e76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:30:04.808948 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349e0a4-114e-4fb5-a6d9-603466615e76 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:30:04.809225 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349e0a4-114e-4fb5-a6d9-603466615e76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:30:04.809497 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349e0a4-114e-4fb5-a6d9-603466615e76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:30:04.811304 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7349e0a4-114e-4fb5-a6d9-603466615e76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:30:04.875205 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7349e0a4-114e-4fb5-a6d9-603466615e76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:30:04.876655 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7349e0a4-114e-4fb5-a6d9-603466615e76 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:30:04.886554 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1093/2185] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:30:04 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:30:13.649741 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=25 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:30:14.883225 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=50 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:30:27.351919 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719baaf9-d673-4ec8-8925-bfbe0fe27c52 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:30:27.352424 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719baaf9-d673-4ec8-8925-bfbe0fe27c52 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:30:27.352542 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719baaf9-d673-4ec8-8925-bfbe0fe27c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:30:27.373876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-719baaf9-d673-4ec8-8925-bfbe0fe27c52 None None] MFA Rules not processed for user `52fc3ba90dc045c2a5b7bb730da96acc`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:30:27.375121 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-719baaf9-d673-4ec8-8925-bfbe0fe27c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:30:27.437562 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1093/2186] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 5 10:30:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:30:27.464075 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7349e0a4-114e-4fb5-a6d9-603466615e76 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:30:27.466585 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7349e0a4-114e-4fb5-a6d9-603466615e76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:30:27.562494 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7349e0a4-114e-4fb5-a6d9-603466615e76 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:30:27.571969 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ef6e96-1e77-46ea-b74e-adad24b1cb0a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c80cf0e429f04f229aea1aa9293a93b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:30:27.573511 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef6e96-1e77-46ea-b74e-adad24b1cb0a service swift] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:30:27.574021 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef6e96-1e77-46ea-b74e-adad24b1cb0a service swift] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:30:27.574389 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef6e96-1e77-46ea-b74e-adad24b1cb0a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:30:27.584579 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ef6e96-1e77-46ea-b74e-adad24b1cb0a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:30:27.586480 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ef6e96-1e77-46ea-b74e-adad24b1cb0a service swift] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:30:27.602381 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1094/2187] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 5 10:30:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 10:30:28.758599 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719baaf9-d673-4ec8-8925-bfbe0fe27c52 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:30:28.855585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719baaf9-d673-4ec8-8925-bfbe0fe27c52 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:30:28.861870 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969088d0-bea2-4360-b95a-97b56681090c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:30:28.862228 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969088d0-bea2-4360-b95a-97b56681090c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:30:28.862532 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969088d0-bea2-4360-b95a-97b56681090c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:30:28.863079 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969088d0-bea2-4360-b95a-97b56681090c admin admin] PATH_INFO: `/v3/users/23c584397b454762a7001e6eb88203ca` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:30:28.864554 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969088d0-bea2-4360-b95a-97b56681090c admin admin] RBAC: Authorizing `identity:delete_user(user_id=23c584397b454762a7001e6eb88203ca)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:30:28.866510 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969088d0-bea2-4360-b95a-97b56681090c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:30:28.948780 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-969088d0-bea2-4360-b95a-97b56681090c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23c584397b454762a7001e6eb88203ca'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:30:28.975540 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-969088d0-bea2-4360-b95a-97b56681090c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23c584397b454762a7001e6eb88203ca'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:30:28.976344 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-969088d0-bea2-4360-b95a-97b56681090c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23c584397b454762a7001e6eb88203ca'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:30:29.003628 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-969088d0-bea2-4360-b95a-97b56681090c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23c584397b454762a7001e6eb88203ca'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:30:29.004408 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-969088d0-bea2-4360-b95a-97b56681090c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23c584397b454762a7001e6eb88203ca'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:30:29.020464 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1094/2188] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:30:28 2022] DELETE /identity/v3/users/23c584397b454762a7001e6eb88203ca => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:30:29.025020 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1116d9-bdb6-49d9-868a-e7c1a7d8a32c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:30:29.025872 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1116d9-bdb6-49d9-868a-e7c1a7d8a32c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:30:29.026152 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1116d9-bdb6-49d9-868a-e7c1a7d8a32c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:30:29.048051 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-cd1116d9-bdb6-49d9-868a-e7c1a7d8a32c None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:30:29.050377 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cd1116d9-bdb6-49d9-868a-e7c1a7d8a32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:30:29.146290 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969088d0-bea2-4360-b95a-97b56681090c admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:30:29.147437 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1095/2189] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:30:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:30:29.149182 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-969088d0-bea2-4360-b95a-97b56681090c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:30:29.206020 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:30:29.246400 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969088d0-bea2-4360-b95a-97b56681090c admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:30:29.252795 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5065f38-2b78-44bd-986f-92c7bb95421a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:30:29.253715 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5065f38-2b78-44bd-986f-92c7bb95421a service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:30:29.254149 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5065f38-2b78-44bd-986f-92c7bb95421a service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:30:29.254812 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5065f38-2b78-44bd-986f-92c7bb95421a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:30:29.282142 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5065f38-2b78-44bd-986f-92c7bb95421a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:30:29.284462 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5065f38-2b78-44bd-986f-92c7bb95421a service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:30:29.323857 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1095/2190] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:30:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:30:29.689182 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1116d9-bdb6-49d9-868a-e7c1a7d8a32c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:30:29.690850 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cd1116d9-bdb6-49d9-868a-e7c1a7d8a32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:30:29.714200 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1116d9-bdb6-49d9-868a-e7c1a7d8a32c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:30:29.728837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2b5c26-f119-48f6-ab8d-4659757e41fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:30:29.729857 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b5c26-f119-48f6-ab8d-4659757e41fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:30:29.730212 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b5c26-f119-48f6-ab8d-4659757e41fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:30:29.730487 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b5c26-f119-48f6-ab8d-4659757e41fd admin admin] PATH_INFO: `/v3/projects/3461d1c72706452b85c97019b722bcf8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:30:29.733033 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2b5c26-f119-48f6-ab8d-4659757e41fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=3461d1c72706452b85c97019b722bcf8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:30:29.735150 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2b5c26-f119-48f6-ab8d-4659757e41fd admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:30:29.835409 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fd2b5c26-f119-48f6-ab8d-4659757e41fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3461d1c72706452b85c97019b722bcf8', 'request_id': 'req-fd2b5c26-f119-48f6-ab8d-4659757e41fd'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:30:29.857902 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fd2b5c26-f119-48f6-ab8d-4659757e41fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3461d1c72706452b85c97019b722bcf8', 'request_id': 'req-fd2b5c26-f119-48f6-ab8d-4659757e41fd'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:30:29.871250 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fd2b5c26-f119-48f6-ab8d-4659757e41fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3461d1c72706452b85c97019b722bcf8', 'request_id': 'req-fd2b5c26-f119-48f6-ab8d-4659757e41fd'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:30:29.887220 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fd2b5c26-f119-48f6-ab8d-4659757e41fd admin admin] The token cache is being invalidate because project 3461d1c72706452b85c97019b722bcf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:30:29.887502 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fd2b5c26-f119-48f6-ab8d-4659757e41fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:30:29.888282 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fd2b5c26-f119-48f6-ab8d-4659757e41fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:30:29.898971 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1096/2191] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:30:29 2022] DELETE /identity/v3/projects/3461d1c72706452b85c97019b722bcf8 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:30:39.297740 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=15 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:30:39.883910 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:30:41.734862 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e200780-b117-4097-ab11-9269220c6661 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:30:41.735424 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e200780-b117-4097-ab11-9269220c6661 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:30:41.735539 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e200780-b117-4097-ab11-9269220c6661 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:30:41.737044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1096/2192] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 5 10:30:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:30:41.760101 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5521a8-d1ba-4b9e-87fe-1b9d1e5643db None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:30:41.760579 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5521a8-d1ba-4b9e-87fe-1b9d1e5643db None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:30:41.760683 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5521a8-d1ba-4b9e-87fe-1b9d1e5643db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:30:41.791020 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-2e5521a8-d1ba-4b9e-87fe-1b9d1e5643db None None] MFA Rules not processed for user `b938f1f41db94b0eafe95bc8eeaae175`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:30:41.793118 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2e5521a8-d1ba-4b9e-87fe-1b9d1e5643db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:30:41.869575 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1097/2193] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 5 10:30:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:30:41.887244 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e200780-b117-4097-ab11-9269220c6661 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:30:41.889562 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6e200780-b117-4097-ab11-9269220c6661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:30:41.954110 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e200780-b117-4097-ab11-9269220c6661 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:30:41.958235 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faecf212-407a-4d0b-aef6-93cabda7fd51 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac0a8cf7f8b74e88a04444ec38547528', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:30:41.958861 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faecf212-407a-4d0b-aef6-93cabda7fd51 service glance] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:30:41.959170 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faecf212-407a-4d0b-aef6-93cabda7fd51 service glance] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:30:41.959444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faecf212-407a-4d0b-aef6-93cabda7fd51 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:30:41.969822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faecf212-407a-4d0b-aef6-93cabda7fd51 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:30:41.971305 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faecf212-407a-4d0b-aef6-93cabda7fd51 service glance] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:30:41.983903 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1097/2194] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:30:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:30:42.063900 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7406dd-84f2-4fc2-ac06-821ad9871908 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:30:42.065576 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7406dd-84f2-4fc2-ac06-821ad9871908 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:30:42.065576 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7406dd-84f2-4fc2-ac06-821ad9871908 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:30:42.065980 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1098/2195] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 5 10:30:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:30:42.071550 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e56ed26-19ab-448b-a7d7-0fef10512b9a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:30:42.071876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e56ed26-19ab-448b-a7d7-0fef10512b9a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:30:42.072148 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e56ed26-19ab-448b-a7d7-0fef10512b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:30:42.095957 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-8e56ed26-19ab-448b-a7d7-0fef10512b9a None None] MFA Rules not processed for user `a7deb94069a449269cb03406232174d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:30:42.097399 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8e56ed26-19ab-448b-a7d7-0fef10512b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:30:42.165525 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1098/2196] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 5 10:30:42 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:30:42.177596 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7406dd-84f2-4fc2-ac06-821ad9871908 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:30:42.179382 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4f7406dd-84f2-4fc2-ac06-821ad9871908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:30:42.195364 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7406dd-84f2-4fc2-ac06-821ad9871908 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:30:42.198869 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d6c387-59c7-4c31-8d77-b2acb17a06ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:30:42.199546 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6c387-59c7-4c31-8d77-b2acb17a06ce service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:30:42.199820 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6c387-59c7-4c31-8d77-b2acb17a06ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:30:42.200092 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6c387-59c7-4c31-8d77-b2acb17a06ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:30:42.208804 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d6c387-59c7-4c31-8d77-b2acb17a06ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:30:42.210254 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d6c387-59c7-4c31-8d77-b2acb17a06ce service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:30:42.218718 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1099/2197] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:30:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:30:42.308709 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf13b144-dd00-419a-864d-5831d74b64b5 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:30:42.310163 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf13b144-dd00-419a-864d-5831d74b64b5 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:30:42.310163 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf13b144-dd00-419a-864d-5831d74b64b5 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:30:42.310799 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1099/2198] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 5 10:30:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:30:42.315919 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba5088-c7ef-45d1-a64c-d1b9b42f9661 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:30:42.316187 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba5088-c7ef-45d1-a64c-d1b9b42f9661 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:30:42.316464 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba5088-c7ef-45d1-a64c-d1b9b42f9661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:30:42.336532 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c2ba5088-c7ef-45d1-a64c-d1b9b42f9661 None None] MFA Rules not processed for user `b938f1f41db94b0eafe95bc8eeaae175`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:30:42.337718 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c2ba5088-c7ef-45d1-a64c-d1b9b42f9661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:30:42.347317 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1100/2199] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 5 10:30:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:30:42.358038 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf13b144-dd00-419a-864d-5831d74b64b5 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:30:42.369270 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf13b144-dd00-419a-864d-5831d74b64b5 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:30:42.372848 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de2051a-d047-4578-b9e9-3e4d1c62f4eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:30:42.373461 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2051a-d047-4578-b9e9-3e4d1c62f4eb service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:30:42.373739 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2051a-d047-4578-b9e9-3e4d1c62f4eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:30:42.374011 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2051a-d047-4578-b9e9-3e4d1c62f4eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:30:42.384198 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de2051a-d047-4578-b9e9-3e4d1c62f4eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:30:42.385676 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de2051a-d047-4578-b9e9-3e4d1c62f4eb service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:30:42.397115 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1100/2200] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:30:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:30:45.507852 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ac29f-6f4f-41aa-ac33-f3bccc90175c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:30:45.508359 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ac29f-6f4f-41aa-ac33-f3bccc90175c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:30:45.508359 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ac29f-6f4f-41aa-ac33-f3bccc90175c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:30:45.529155 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ec6ac29f-6f4f-41aa-ac33-f3bccc90175c None None] MFA Rules not processed for user `52fc3ba90dc045c2a5b7bb730da96acc`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:30:45.530684 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ec6ac29f-6f4f-41aa-ac33-f3bccc90175c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:30:45.586037 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1101/2201] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 5 10:30:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:30:45.599063 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de2051a-d047-4578-b9e9-3e4d1c62f4eb service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:30:45.600742 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5de2051a-d047-4578-b9e9-3e4d1c62f4eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:30:45.675737 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de2051a-d047-4578-b9e9-3e4d1c62f4eb service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:30:45.680989 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3033870-4136-4ef6-ad13-f230c0e222e7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c80cf0e429f04f229aea1aa9293a93b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:30:45.681611 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3033870-4136-4ef6-ad13-f230c0e222e7 service swift] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:30:45.681894 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3033870-4136-4ef6-ad13-f230c0e222e7 service swift] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:30:45.682162 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3033870-4136-4ef6-ad13-f230c0e222e7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:30:45.690446 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3033870-4136-4ef6-ad13-f230c0e222e7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:30:45.692153 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3033870-4136-4ef6-ad13-f230c0e222e7 service swift] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:30:45.700725 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1101/2202] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 5 10:30:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 10:30:55.578674 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=65 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:30:55.697020 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=87 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:31:06.245824 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ac29f-6f4f-41aa-ac33-f3bccc90175c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:31:06.247464 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ec6ac29f-6f4f-41aa-ac33-f3bccc90175c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:31:06.261031 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ac29f-6f4f-41aa-ac33-f3bccc90175c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:31:06.266712 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2549ed5-2cd6-44af-a7f3-42c543b396a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:31:06.267680 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2549ed5-2cd6-44af-a7f3-42c543b396a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:31:06.268035 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2549ed5-2cd6-44af-a7f3-42c543b396a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:31:06.268417 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2549ed5-2cd6-44af-a7f3-42c543b396a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:31:06.271632 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a2549ed5-2cd6-44af-a7f3-42c543b396a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:31:06.286970 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2549ed5-2cd6-44af-a7f3-42c543b396a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:31:06.288485 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2549ed5-2cd6-44af-a7f3-42c543b396a9 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:31:06.299835 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1102/2203] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:31:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:31:08.430031 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3033870-4136-4ef6-ad13-f230c0e222e7 service swift] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:31:08.440805 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3033870-4136-4ef6-ad13-f230c0e222e7 service swift] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:31:08.445712 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba9a409-5aed-4ff4-9492-52ee99b25f58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:31:08.446309 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9a409-5aed-4ff4-9492-52ee99b25f58 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:31:08.446615 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9a409-5aed-4ff4-9492-52ee99b25f58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:31:08.446879 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9a409-5aed-4ff4-9492-52ee99b25f58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:31:08.448931 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-dba9a409-5aed-4ff4-9492-52ee99b25f58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:31:08.463521 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba9a409-5aed-4ff4-9492-52ee99b25f58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:31:08.464956 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba9a409-5aed-4ff4-9492-52ee99b25f58 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:31:08.475966 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1102/2204] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:31:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:31:16.294844 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=6 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:31:18.470889 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=6 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:31:20.480712 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2549ed5-2cd6-44af-a7f3-42c543b396a9 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:31:20.482277 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a2549ed5-2cd6-44af-a7f3-42c543b396a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:31:20.545501 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2549ed5-2cd6-44af-a7f3-42c543b396a9 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:31:20.549615 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91402df-6b0b-4ac9-80b6-a14b52fe319f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:31:20.550247 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91402df-6b0b-4ac9-80b6-a14b52fe319f service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:31:20.550529 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91402df-6b0b-4ac9-80b6-a14b52fe319f service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:31:20.550810 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91402df-6b0b-4ac9-80b6-a14b52fe319f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:31:20.552806 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b91402df-6b0b-4ac9-80b6-a14b52fe319f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:31:20.615739 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91402df-6b0b-4ac9-80b6-a14b52fe319f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:31:20.617199 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91402df-6b0b-4ac9-80b6-a14b52fe319f service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:31:20.626646 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1103/2205] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:31:20 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:31:23.751296 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adaa881-30e9-492d-88a0-e03380d5a814 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:31:23.751761 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adaa881-30e9-492d-88a0-e03380d5a814 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:31:23.751849 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adaa881-30e9-492d-88a0-e03380d5a814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:31:23.772290 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-8adaa881-30e9-492d-88a0-e03380d5a814 None None] MFA Rules not processed for user `52fc3ba90dc045c2a5b7bb730da96acc`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:31:23.773486 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8adaa881-30e9-492d-88a0-e03380d5a814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:31:23.782383 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1103/2206] 10.222.0.40 () {58 vars in 967 bytes} [Fri Aug 5 10:31:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:31:23.794697 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91402df-6b0b-4ac9-80b6-a14b52fe319f service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:31:23.807285 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91402df-6b0b-4ac9-80b6-a14b52fe319f service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:31:23.812978 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3950427-41a4-4354-99fa-49a56ae96312 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c80cf0e429f04f229aea1aa9293a93b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:31:23.813591 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3950427-41a4-4354-99fa-49a56ae96312 service swift] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:31:23.813866 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3950427-41a4-4354-99fa-49a56ae96312 service swift] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:31:23.814260 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3950427-41a4-4354-99fa-49a56ae96312 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:31:23.824056 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3950427-41a4-4354-99fa-49a56ae96312 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:31:23.825756 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3950427-41a4-4354-99fa-49a56ae96312 service swift] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:31:23.833821 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1104/2207] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 5 10:31:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 10:31:33.767030 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:31:33.830777 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=54 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:31:39.249559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adaa881-30e9-492d-88a0-e03380d5a814 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:31:39.252097 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8adaa881-30e9-492d-88a0-e03380d5a814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:31:39.273685 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adaa881-30e9-492d-88a0-e03380d5a814 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:31:39.280557 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76848b3f-d3d0-4420-9a63-39c26f173983 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:31:39.281207 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76848b3f-d3d0-4420-9a63-39c26f173983 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:31:39.281507 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76848b3f-d3d0-4420-9a63-39c26f173983 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:31:39.281773 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76848b3f-d3d0-4420-9a63-39c26f173983 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:31:39.283750 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-76848b3f-d3d0-4420-9a63-39c26f173983 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:31:39.345365 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76848b3f-d3d0-4420-9a63-39c26f173983 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:31:39.346867 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76848b3f-d3d0-4420-9a63-39c26f173983 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:31:39.357371 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1104/2208] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:31:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:31:44.797600 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3950427-41a4-4354-99fa-49a56ae96312 service swift] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:31:44.807994 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3950427-41a4-4354-99fa-49a56ae96312 service swift] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:31:44.813057 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3004c6-4514-45ac-b770-317ef5ec1a74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:31:44.813683 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3004c6-4514-45ac-b770-317ef5ec1a74 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:31:44.813960 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3004c6-4514-45ac-b770-317ef5ec1a74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:31:44.814231 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3004c6-4514-45ac-b770-317ef5ec1a74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:31:44.816343 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9f3004c6-4514-45ac-b770-317ef5ec1a74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:31:44.826672 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3004c6-4514-45ac-b770-317ef5ec1a74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:31:44.828154 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3004c6-4514-45ac-b770-317ef5ec1a74 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:31:44.836413 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1105/2209] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:31:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:31:47.931334 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2a230b-b71b-43b3-9b76-3a0f2d019377 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:31:47.931888 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2a230b-b71b-43b3-9b76-3a0f2d019377 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:31:47.932015 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2a230b-b71b-43b3-9b76-3a0f2d019377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:31:47.952706 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-7e2a230b-b71b-43b3-9b76-3a0f2d019377 None None] MFA Rules not processed for user `a3b4d90bcfc34d2d9ba1c40316e09858`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:31:47.953925 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7e2a230b-b71b-43b3-9b76-3a0f2d019377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:31:47.961387 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1105/2210] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:31:47 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:31:54.395317 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3004c6-4514-45ac-b770-317ef5ec1a74 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:31:54.396907 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9f3004c6-4514-45ac-b770-317ef5ec1a74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:31:54.419699 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3004c6-4514-45ac-b770-317ef5ec1a74 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:31:54.426209 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f730351-034d-4ec1-8769-35055abb41e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:31:54.426960 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f730351-034d-4ec1-8769-35055abb41e3 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:31:54.427326 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f730351-034d-4ec1-8769-35055abb41e3 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:31:54.427606 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f730351-034d-4ec1-8769-35055abb41e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:31:54.429467 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4f730351-034d-4ec1-8769-35055abb41e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:31:54.442292 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f730351-034d-4ec1-8769-35055abb41e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:31:54.443927 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f730351-034d-4ec1-8769-35055abb41e3 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:31:54.454679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1106/2211] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:31:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:31:57.946789 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=31 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:32:04.450333 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=12 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:32:06.389992 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2a230b-b71b-43b3-9b76-3a0f2d019377 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:32:06.403311 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2a230b-b71b-43b3-9b76-3a0f2d019377 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:32:06.408921 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adad6cc-2225-4755-aa73-c886ccb6932f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:32:06.409544 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adad6cc-2225-4755-aa73-c886ccb6932f service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:32:06.409818 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adad6cc-2225-4755-aa73-c886ccb6932f service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:32:06.410085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adad6cc-2225-4755-aa73-c886ccb6932f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:32:06.412114 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3adad6cc-2225-4755-aa73-c886ccb6932f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:32:06.471637 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adad6cc-2225-4755-aa73-c886ccb6932f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:32:06.474281 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adad6cc-2225-4755-aa73-c886ccb6932f service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:32:06.489573 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1106/2212] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:32:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 10:32:09.909827 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f730351-034d-4ec1-8769-35055abb41e3 service nova] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:32:09.924368 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f730351-034d-4ec1-8769-35055abb41e3 service nova] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:32:09.931244 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd875a-b8e1-400e-948f-4eefc214b5dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:32:09.931930 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd875a-b8e1-400e-948f-4eefc214b5dc service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:32:09.932272 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd875a-b8e1-400e-948f-4eefc214b5dc service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:32:09.932606 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd875a-b8e1-400e-948f-4eefc214b5dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:32:09.934606 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-90dd875a-b8e1-400e-948f-4eefc214b5dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:32:09.947920 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dd875a-b8e1-400e-948f-4eefc214b5dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:32:09.950003 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dd875a-b8e1-400e-948f-4eefc214b5dc service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:32:09.962087 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1107/2213] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:32:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:32:16.482113 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=28 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:32:19.957342 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=6 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:32:20.774500 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceced5e-af72-48e3-8c08-ca808b5a5e27 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:32:20.775043 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceced5e-af72-48e3-8c08-ca808b5a5e27 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:32:20.775043 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceced5e-af72-48e3-8c08-ca808b5a5e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:32:20.795648 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-5ceced5e-af72-48e3-8c08-ca808b5a5e27 None None] MFA Rules not processed for user `706dbfa719c04fd0a3704ebd7eb812cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:32:20.796853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5ceced5e-af72-48e3-8c08-ca808b5a5e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:32:20.804261 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1107/2214] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:32:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:32:21.128739 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd875a-b8e1-400e-948f-4eefc214b5dc service nova] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:32:21.138927 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd875a-b8e1-400e-948f-4eefc214b5dc service nova] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:32:21.143850 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0378880-f548-45a5-a3e2-f0dfde92501f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:32:21.144457 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0378880-f548-45a5-a3e2-f0dfde92501f service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:32:21.144732 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0378880-f548-45a5-a3e2-f0dfde92501f service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:32:21.145001 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0378880-f548-45a5-a3e2-f0dfde92501f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:32:21.147140 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e0378880-f548-45a5-a3e2-f0dfde92501f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:32:21.160251 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0378880-f548-45a5-a3e2-f0dfde92501f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:32:21.161687 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0378880-f548-45a5-a3e2-f0dfde92501f service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:32:21.170161 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1108/2215] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:32:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:32:21.500638 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceced5e-af72-48e3-8c08-ca808b5a5e27 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:32:21.502304 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5ceced5e-af72-48e3-8c08-ca808b5a5e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:32:21.575156 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceced5e-af72-48e3-8c08-ca808b5a5e27 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:32:21.581874 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c221ad57-49dd-4460-8c90-5f46293f2805 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8743f5bb0be54d57a957617636b0b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:32:21.582511 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c221ad57-49dd-4460-8c90-5f46293f2805 service placement] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:32:21.582780 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c221ad57-49dd-4460-8c90-5f46293f2805 service placement] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:32:21.583097 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c221ad57-49dd-4460-8c90-5f46293f2805 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:32:21.585282 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c221ad57-49dd-4460-8c90-5f46293f2805 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:32:21.598184 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c221ad57-49dd-4460-8c90-5f46293f2805 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:32:21.599328 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c221ad57-49dd-4460-8c90-5f46293f2805 service placement] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:32:21.609759 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1108/2216] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:32:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:32:30.900526 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0378880-f548-45a5-a3e2-f0dfde92501f service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:32:30.924076 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0378880-f548-45a5-a3e2-f0dfde92501f service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:32:30.936893 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42aae1-4167-4350-8600-63fe194982a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:32:30.938037 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42aae1-4167-4350-8600-63fe194982a2 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:32:30.938467 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42aae1-4167-4350-8600-63fe194982a2 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:32:30.938886 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42aae1-4167-4350-8600-63fe194982a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:32:30.943225 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-df42aae1-4167-4350-8600-63fe194982a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:32:30.959396 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df42aae1-4167-4350-8600-63fe194982a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:32:30.961528 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df42aae1-4167-4350-8600-63fe194982a2 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:32:30.992455 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1109/2217] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:32:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:32:31.605218 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=35 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:32:32.690839 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c221ad57-49dd-4460-8c90-5f46293f2805 service placement] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:32:32.703367 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c221ad57-49dd-4460-8c90-5f46293f2805 service placement] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:32:32.710333 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22098b2c-5289-4420-ab7e-edd992f063a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:32:32.711084 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22098b2c-5289-4420-ab7e-edd992f063a0 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:32:32.711425 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22098b2c-5289-4420-ab7e-edd992f063a0 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:32:32.711760 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22098b2c-5289-4420-ab7e-edd992f063a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:32:32.713583 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-22098b2c-5289-4420-ab7e-edd992f063a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:32:32.771453 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22098b2c-5289-4420-ab7e-edd992f063a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:32:32.773030 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22098b2c-5289-4420-ab7e-edd992f063a0 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:32:32.783063 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1109/2218] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:32:32 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:32:40.972436 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=12 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:32:42.138597 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42aae1-4167-4350-8600-63fe194982a2 service nova] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:32:42.140092 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-df42aae1-4167-4350-8600-63fe194982a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:32:42.152571 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42aae1-4167-4350-8600-63fe194982a2 service nova] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:32:42.156855 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0669d8-aca1-4fe4-b556-326bcd84f6db service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac0a8cf7f8b74e88a04444ec38547528', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:32:42.157932 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0669d8-aca1-4fe4-b556-326bcd84f6db service glance] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:32:42.157932 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0669d8-aca1-4fe4-b556-326bcd84f6db service glance] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:32:42.158127 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0669d8-aca1-4fe4-b556-326bcd84f6db service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:32:42.159851 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5c0669d8-aca1-4fe4-b556-326bcd84f6db service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:32:42.231202 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0669d8-aca1-4fe4-b556-326bcd84f6db service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:32:42.232725 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0669d8-aca1-4fe4-b556-326bcd84f6db service glance] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:32:42.243623 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1110/2219] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:32:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:32:42.271482 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ffb79b-31a4-49be-9c49-3c5dae9d4ae9 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:32:42.272091 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ffb79b-31a4-49be-9c49-3c5dae9d4ae9 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:32:42.272549 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ffb79b-31a4-49be-9c49-3c5dae9d4ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:32:42.294873 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-d3ffb79b-31a4-49be-9c49-3c5dae9d4ae9 None None] MFA Rules not processed for user `52fc3ba90dc045c2a5b7bb730da96acc`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:32:42.296142 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d3ffb79b-31a4-49be-9c49-3c5dae9d4ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:32:42.303815 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1110/2220] 10.222.0.40 () {58 vars in 967 bytes} [Fri Aug 5 10:32:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:32:42.364615 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0669d8-aca1-4fe4-b556-326bcd84f6db service glance] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:32:42.377528 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0669d8-aca1-4fe4-b556-326bcd84f6db service glance] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:32:42.383898 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476e46d7-231e-4be6-8b2d-973118713565 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c80cf0e429f04f229aea1aa9293a93b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:32:42.384553 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e46d7-231e-4be6-8b2d-973118713565 service swift] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:32:42.384840 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e46d7-231e-4be6-8b2d-973118713565 service swift] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:32:42.385109 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e46d7-231e-4be6-8b2d-973118713565 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:32:42.394763 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476e46d7-231e-4be6-8b2d-973118713565 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:32:42.396447 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476e46d7-231e-4be6-8b2d-973118713565 service swift] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:32:42.406413 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1111/2221] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 5 10:32:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 10:32:42.793658 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ffb79b-31a4-49be-9c49-3c5dae9d4ae9 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:32:42.803647 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ffb79b-31a4-49be-9c49-3c5dae9d4ae9 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:32:42.808482 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f26eab-f6c4-43d7-9838-6b002b69a3dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:32:42.809150 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f26eab-f6c4-43d7-9838-6b002b69a3dc service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:32:42.809443 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f26eab-f6c4-43d7-9838-6b002b69a3dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:32:42.809816 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f26eab-f6c4-43d7-9838-6b002b69a3dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:32:42.811977 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c1f26eab-f6c4-43d7-9838-6b002b69a3dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:32:42.879692 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f26eab-f6c4-43d7-9838-6b002b69a3dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:32:42.881557 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f26eab-f6c4-43d7-9838-6b002b69a3dc service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:32:42.891841 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1111/2222] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:32:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:32:46.207989 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc92a34-a815-4c02-bf05-661b900a4e13 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:32:46.208507 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc92a34-a815-4c02-bf05-661b900a4e13 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:32:46.208507 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc92a34-a815-4c02-bf05-661b900a4e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:32:46.230609 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-adc92a34-a815-4c02-bf05-661b900a4e13 None None] MFA Rules not processed for user `97f671de92d240ae85ca4893d1ff04e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:32:46.231998 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-adc92a34-a815-4c02-bf05-661b900a4e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:32:46.239345 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1112/2223] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:32:46 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:32:47.467654 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f26eab-f6c4-43d7-9838-6b002b69a3dc service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:32:47.469233 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c1f26eab-f6c4-43d7-9838-6b002b69a3dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:32:47.484471 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f26eab-f6c4-43d7-9838-6b002b69a3dc service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:32:47.490853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff674e18-19a5-4c25-b251-3bf270d3f2d1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8743f5bb0be54d57a957617636b0b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:32:47.491572 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff674e18-19a5-4c25-b251-3bf270d3f2d1 service placement] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:32:47.491857 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff674e18-19a5-4c25-b251-3bf270d3f2d1 service placement] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:32:47.492130 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff674e18-19a5-4c25-b251-3bf270d3f2d1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:32:47.494585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ff674e18-19a5-4c25-b251-3bf270d3f2d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:32:47.514915 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff674e18-19a5-4c25-b251-3bf270d3f2d1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:32:47.516749 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff674e18-19a5-4c25-b251-3bf270d3f2d1 service placement] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:32:47.535085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1112/2224] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:32:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:32:51.724002 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc92a34-a815-4c02-bf05-661b900a4e13 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:32:51.739981 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc92a34-a815-4c02-bf05-661b900a4e13 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:32:51.746491 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aefb9f-85e9-44e2-aaec-a6e718cca60e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8743f5bb0be54d57a957617636b0b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:32:51.747163 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aefb9f-85e9-44e2-aaec-a6e718cca60e service placement] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:32:51.747437 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aefb9f-85e9-44e2-aaec-a6e718cca60e service placement] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:32:51.747710 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aefb9f-85e9-44e2-aaec-a6e718cca60e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:32:51.749630 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e6aefb9f-85e9-44e2-aaec-a6e718cca60e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:32:51.762549 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aefb9f-85e9-44e2-aaec-a6e718cca60e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:32:51.764201 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aefb9f-85e9-44e2-aaec-a6e718cca60e service placement] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:32:51.775688 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1113/2225] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:32:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:32:52.082415 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff674e18-19a5-4c25-b251-3bf270d3f2d1 service placement] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:32:52.083982 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ff674e18-19a5-4c25-b251-3bf270d3f2d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:32:52.100164 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff674e18-19a5-4c25-b251-3bf270d3f2d1 service placement] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:32:52.106109 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff6462e-8377-4ef5-8ebf-da14dab8bfd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:32:52.106769 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff6462e-8377-4ef5-8ebf-da14dab8bfd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:32:52.107097 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff6462e-8377-4ef5-8ebf-da14dab8bfd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:32:52.107363 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff6462e-8377-4ef5-8ebf-da14dab8bfd8 admin admin] PATH_INFO: `/v3/users/9697af06b1104425aed6692ba4f29238` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:32:52.108665 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff6462e-8377-4ef5-8ebf-da14dab8bfd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9697af06b1104425aed6692ba4f29238)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:32:52.110602 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff6462e-8377-4ef5-8ebf-da14dab8bfd8 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:32:52.170952 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-aff6462e-8377-4ef5-8ebf-da14dab8bfd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9697af06b1104425aed6692ba4f29238'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:32:52.180194 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-aff6462e-8377-4ef5-8ebf-da14dab8bfd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9697af06b1104425aed6692ba4f29238'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:32:52.180871 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-aff6462e-8377-4ef5-8ebf-da14dab8bfd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9697af06b1104425aed6692ba4f29238'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:32:52.190822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:32:52.193142 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-aff6462e-8377-4ef5-8ebf-da14dab8bfd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9697af06b1104425aed6692ba4f29238'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:32:52.194786 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-aff6462e-8377-4ef5-8ebf-da14dab8bfd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9697af06b1104425aed6692ba4f29238'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:32:52.207679 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1113/2226] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:32:52 2022] DELETE /identity/v3/users/9697af06b1104425aed6692ba4f29238 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:32:52.393886 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aefb9f-85e9-44e2-aaec-a6e718cca60e service placement] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:32:52.408564 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aefb9f-85e9-44e2-aaec-a6e718cca60e service placement] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:32:52.476454 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257a0db8-8756-4429-af1d-f6c6f20ea089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:32:52.477065 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a0db8-8756-4429-af1d-f6c6f20ea089 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:32:52.477436 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a0db8-8756-4429-af1d-f6c6f20ea089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:32:52.477705 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a0db8-8756-4429-af1d-f6c6f20ea089 admin admin] PATH_INFO: `/v3/projects/db2897585ac048ae93df3e38d1437e6e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:32:52.479281 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257a0db8-8756-4429-af1d-f6c6f20ea089 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db2897585ac048ae93df3e38d1437e6e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:32:52.481920 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257a0db8-8756-4429-af1d-f6c6f20ea089 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:32:52.540550 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-257a0db8-8756-4429-af1d-f6c6f20ea089 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db2897585ac048ae93df3e38d1437e6e', 'request_id': 'req-257a0db8-8756-4429-af1d-f6c6f20ea089'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:32:52.552000 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-257a0db8-8756-4429-af1d-f6c6f20ea089 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db2897585ac048ae93df3e38d1437e6e', 'request_id': 'req-257a0db8-8756-4429-af1d-f6c6f20ea089'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:32:52.557814 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-257a0db8-8756-4429-af1d-f6c6f20ea089 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db2897585ac048ae93df3e38d1437e6e', 'request_id': 'req-257a0db8-8756-4429-af1d-f6c6f20ea089'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:32:52.559743 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:32:52.571212 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-257a0db8-8756-4429-af1d-f6c6f20ea089 admin admin] The token cache is being invalidate because project db2897585ac048ae93df3e38d1437e6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:32:52.571504 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-257a0db8-8756-4429-af1d-f6c6f20ea089 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:32:52.572480 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-257a0db8-8756-4429-af1d-f6c6f20ea089 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:32:52.574229 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1114/2227] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:32:52 2022] DELETE /identity/v3/projects/db2897585ac048ae93df3e38d1437e6e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:32:52.578572 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff6462e-8377-4ef5-8ebf-da14dab8bfd8 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:32:52.580071 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-aff6462e-8377-4ef5-8ebf-da14dab8bfd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:32:52.597715 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff6462e-8377-4ef5-8ebf-da14dab8bfd8 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:32:52.668527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8322010d-3815-4e3f-89d2-ae050dfb1205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:32:52.669177 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8322010d-3815-4e3f-89d2-ae050dfb1205 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:32:52.669456 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8322010d-3815-4e3f-89d2-ae050dfb1205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:32:52.669720 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8322010d-3815-4e3f-89d2-ae050dfb1205 admin admin] PATH_INFO: `/v3/users/3a93ce5d2392424f80afe18980c8a37e` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:32:52.670995 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8322010d-3815-4e3f-89d2-ae050dfb1205 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a93ce5d2392424f80afe18980c8a37e)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:32:52.672944 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8322010d-3815-4e3f-89d2-ae050dfb1205 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:32:52.725430 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8322010d-3815-4e3f-89d2-ae050dfb1205 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a93ce5d2392424f80afe18980c8a37e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:32:52.734284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8322010d-3815-4e3f-89d2-ae050dfb1205 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a93ce5d2392424f80afe18980c8a37e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:32:52.734852 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8322010d-3815-4e3f-89d2-ae050dfb1205 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a93ce5d2392424f80afe18980c8a37e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:32:52.746779 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8322010d-3815-4e3f-89d2-ae050dfb1205 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a93ce5d2392424f80afe18980c8a37e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:32:52.747517 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8322010d-3815-4e3f-89d2-ae050dfb1205 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a93ce5d2392424f80afe18980c8a37e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:32:52.759924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1114/2228] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:32:52 2022] DELETE /identity/v3/users/3a93ce5d2392424f80afe18980c8a37e => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:32:52.941999 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257a0db8-8756-4429-af1d-f6c6f20ea089 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:32:52.943826 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-257a0db8-8756-4429-af1d-f6c6f20ea089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:32:52.959340 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257a0db8-8756-4429-af1d-f6c6f20ea089 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:32:53.025634 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a4c2b-2799-49c5-97ef-44fded0bb537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:32:53.026190 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a4c2b-2799-49c5-97ef-44fded0bb537 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:32:53.026470 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a4c2b-2799-49c5-97ef-44fded0bb537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:32:53.026747 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a4c2b-2799-49c5-97ef-44fded0bb537 admin admin] PATH_INFO: `/v3/projects/7a197ca65c7a440395ab38c607d766f1` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:32:53.028018 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175a4c2b-2799-49c5-97ef-44fded0bb537 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a197ca65c7a440395ab38c607d766f1)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:32:53.030731 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175a4c2b-2799-49c5-97ef-44fded0bb537 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:32:53.093830 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-175a4c2b-2799-49c5-97ef-44fded0bb537 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a197ca65c7a440395ab38c607d766f1', 'request_id': 'req-175a4c2b-2799-49c5-97ef-44fded0bb537'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:32:53.105076 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-175a4c2b-2799-49c5-97ef-44fded0bb537 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a197ca65c7a440395ab38c607d766f1', 'request_id': 'req-175a4c2b-2799-49c5-97ef-44fded0bb537'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:32:53.110850 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-175a4c2b-2799-49c5-97ef-44fded0bb537 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a197ca65c7a440395ab38c607d766f1', 'request_id': 'req-175a4c2b-2799-49c5-97ef-44fded0bb537'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:32:53.122527 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-175a4c2b-2799-49c5-97ef-44fded0bb537 admin admin] The token cache is being invalidate because project 7a197ca65c7a440395ab38c607d766f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:32:53.122815 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-175a4c2b-2799-49c5-97ef-44fded0bb537 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:32:53.123481 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-175a4c2b-2799-49c5-97ef-44fded0bb537 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:32:53.124999 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1115/2229] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:32:52 2022] DELETE /identity/v3/projects/7a197ca65c7a440395ab38c607d766f1 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:33:02.756481 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:DELETE=11,keystone:SELECT=59,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:33:03.119484 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=53,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:33:18.890813 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8322010d-3815-4e3f-89d2-ae050dfb1205 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:18.893012 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8322010d-3815-4e3f-89d2-ae050dfb1205 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:19.020762 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8322010d-3815-4e3f-89d2-ae050dfb1205 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:19.096414 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f46f48c-5531-437b-89bb-27dc1902419e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:19.097051 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46f48c-5531-437b-89bb-27dc1902419e service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:19.097330 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46f48c-5531-437b-89bb-27dc1902419e service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:19.097618 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46f48c-5531-437b-89bb-27dc1902419e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:19.099462 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9f46f48c-5531-437b-89bb-27dc1902419e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:19.188046 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f46f48c-5531-437b-89bb-27dc1902419e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:19.189592 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f46f48c-5531-437b-89bb-27dc1902419e service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:19.200096 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1115/2230] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:33:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:33:20.481978 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5e434-ad58-4075-9629-75fc7171f4b4 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:20.483023 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5e434-ad58-4075-9629-75fc7171f4b4 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:20.483485 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5e434-ad58-4075-9629-75fc7171f4b4 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:20.485628 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1116/2231] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 5 10:33:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 10:33:20.491727 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7726b0-9d90-443a-a2f9-636ba795b56d None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:20.492247 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7726b0-9d90-443a-a2f9-636ba795b56d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:20.492344 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7726b0-9d90-443a-a2f9-636ba795b56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:20.520113 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-4a7726b0-9d90-443a-a2f9-636ba795b56d None None] MFA Rules not processed for user `a7deb94069a449269cb03406232174d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:33:20.521375 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4a7726b0-9d90-443a-a2f9-636ba795b56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:20.539356 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1116/2232] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 5 10:33:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:33:20.572509 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea5e434-ad58-4075-9629-75fc7171f4b4 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:20.574676 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3ea5e434-ad58-4075-9629-75fc7171f4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:20.597085 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea5e434-ad58-4075-9629-75fc7171f4b4 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:20.604234 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65129701-dbbf-4efe-bf93-d583d4e4dcba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:20.606758 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65129701-dbbf-4efe-bf93-d583d4e4dcba service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:20.607131 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65129701-dbbf-4efe-bf93-d583d4e4dcba service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:20.607411 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65129701-dbbf-4efe-bf93-d583d4e4dcba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:20.625128 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65129701-dbbf-4efe-bf93-d583d4e4dcba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:20.640907 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65129701-dbbf-4efe-bf93-d583d4e4dcba service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:20.656254 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1117/2233] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:33:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:33:20.852512 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52129c04-2ee8-4d86-8967-02d2ebd70c8c None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:20.852731 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52129c04-2ee8-4d86-8967-02d2ebd70c8c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:20.853305 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52129c04-2ee8-4d86-8967-02d2ebd70c8c None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:20.854867 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1117/2234] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 5 10:33:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 10:33:20.860782 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43063d2b-a096-4fdc-92ec-bc28feb7bf9d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:20.861274 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43063d2b-a096-4fdc-92ec-bc28feb7bf9d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:20.861732 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43063d2b-a096-4fdc-92ec-bc28feb7bf9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:20.884570 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-43063d2b-a096-4fdc-92ec-bc28feb7bf9d None None] MFA Rules not processed for user `b938f1f41db94b0eafe95bc8eeaae175`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:33:20.885787 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-43063d2b-a096-4fdc-92ec-bc28feb7bf9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:20.961459 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1118/2235] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 5 10:33:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:33:20.972518 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52129c04-2ee8-4d86-8967-02d2ebd70c8c None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:20.974200 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-52129c04-2ee8-4d86-8967-02d2ebd70c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:20.987380 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52129c04-2ee8-4d86-8967-02d2ebd70c8c None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:20.991053 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500bbdb4-ff32-4eb7-9e5b-5970106c30c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:20.991668 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500bbdb4-ff32-4eb7-9e5b-5970106c30c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:20.991949 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500bbdb4-ff32-4eb7-9e5b-5970106c30c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:20.992217 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500bbdb4-ff32-4eb7-9e5b-5970106c30c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:21.002610 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500bbdb4-ff32-4eb7-9e5b-5970106c30c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:21.004101 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500bbdb4-ff32-4eb7-9e5b-5970106c30c0 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:21.015122 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1118/2236] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:33:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:33:24.674112 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43063d2b-a096-4fdc-92ec-bc28feb7bf9d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:24.675724 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-43063d2b-a096-4fdc-92ec-bc28feb7bf9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:24.691082 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43063d2b-a096-4fdc-92ec-bc28feb7bf9d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:24.697635 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73102c8d-0013-4231-a925-89a11961e86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:24.698429 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73102c8d-0013-4231-a925-89a11961e86f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:24.698719 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73102c8d-0013-4231-a925-89a11961e86f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:24.699031 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73102c8d-0013-4231-a925-89a11961e86f admin admin] PATH_INFO: `/v3/users/a3b4d90bcfc34d2d9ba1c40316e09858` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:24.700232 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73102c8d-0013-4231-a925-89a11961e86f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3b4d90bcfc34d2d9ba1c40316e09858)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:24.702265 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73102c8d-0013-4231-a925-89a11961e86f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:24.758294 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-73102c8d-0013-4231-a925-89a11961e86f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3b4d90bcfc34d2d9ba1c40316e09858'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:24.758634 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-73102c8d-0013-4231-a925-89a11961e86f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3b4d90bcfc34d2d9ba1c40316e09858'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:24.759127 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-73102c8d-0013-4231-a925-89a11961e86f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3b4d90bcfc34d2d9ba1c40316e09858'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:24.769392 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-73102c8d-0013-4231-a925-89a11961e86f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3b4d90bcfc34d2d9ba1c40316e09858'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:24.777865 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-73102c8d-0013-4231-a925-89a11961e86f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3b4d90bcfc34d2d9ba1c40316e09858'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:24.798756 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1119/2237] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:33:24 2022] DELETE /identity/v3/users/a3b4d90bcfc34d2d9ba1c40316e09858 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:33:24.979906 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500bbdb4-ff32-4eb7-9e5b-5970106c30c0 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:24.983419 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35230c64-6d85-4a5c-a46c-17fe720d70eb None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:24.983774 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35230c64-6d85-4a5c-a46c-17fe720d70eb None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:24.984122 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35230c64-6d85-4a5c-a46c-17fe720d70eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:24.991627 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500bbdb4-ff32-4eb7-9e5b-5970106c30c0 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:25.010027 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-35230c64-6d85-4a5c-a46c-17fe720d70eb None None] MFA Rules not processed for user `7e7eb355fac04252a4be1f8e60037924`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:33:25.011383 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-35230c64-6d85-4a5c-a46c-17fe720d70eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:25.041319 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:33:25.055148 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c17f7-8c56-4db3-a42e-d029d0230c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:25.055800 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c17f7-8c56-4db3-a42e-d029d0230c70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:25.056077 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c17f7-8c56-4db3-a42e-d029d0230c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:25.056348 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c17f7-8c56-4db3-a42e-d029d0230c70 admin admin] PATH_INFO: `/v3/projects/9ea83951557a48fcaab832aea2273453` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:25.057504 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c17f7-8c56-4db3-a42e-d029d0230c70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ea83951557a48fcaab832aea2273453)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:25.059591 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c17f7-8c56-4db3-a42e-d029d0230c70 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:25.064953 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:33:25.078635 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1120/2238] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:33:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:33:25.109372 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6b2c17f7-8c56-4db3-a42e-d029d0230c70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ea83951557a48fcaab832aea2273453', 'request_id': 'req-6b2c17f7-8c56-4db3-a42e-d029d0230c70'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:25.115187 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6b2c17f7-8c56-4db3-a42e-d029d0230c70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ea83951557a48fcaab832aea2273453', 'request_id': 'req-6b2c17f7-8c56-4db3-a42e-d029d0230c70'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:25.124534 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6b2c17f7-8c56-4db3-a42e-d029d0230c70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ea83951557a48fcaab832aea2273453', 'request_id': 'req-6b2c17f7-8c56-4db3-a42e-d029d0230c70'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:25.143032 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6b2c17f7-8c56-4db3-a42e-d029d0230c70 admin admin] The token cache is being invalidate because project 9ea83951557a48fcaab832aea2273453 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:33:25.143032 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6b2c17f7-8c56-4db3-a42e-d029d0230c70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:25.143558 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6b2c17f7-8c56-4db3-a42e-d029d0230c70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:25.145326 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1119/2239] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:33:24 2022] DELETE /identity/v3/projects/9ea83951557a48fcaab832aea2273453 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:33:25.150284 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d01de-37e9-4bd1-80ea-479570f294ff None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:25.150815 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d01de-37e9-4bd1-80ea-479570f294ff None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:25.152646 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d01de-37e9-4bd1-80ea-479570f294ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:25.175231 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-900d01de-37e9-4bd1-80ea-479570f294ff None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:33:25.176647 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-900d01de-37e9-4bd1-80ea-479570f294ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:25.251018 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1121/2240] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:33:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:33:25.255731 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fabf54-fb4d-47cc-9987-92142a8d32ca None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:25.256019 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fabf54-fb4d-47cc-9987-92142a8d32ca None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:25.256298 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fabf54-fb4d-47cc-9987-92142a8d32ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:25.276498 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-15fabf54-fb4d-47cc-9987-92142a8d32ca None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:33:25.278002 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-15fabf54-fb4d-47cc-9987-92142a8d32ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:25.286007 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1120/2241] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:33:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:33:25.290388 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d01de-37e9-4bd1-80ea-479570f294ff None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:25.301222 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d01de-37e9-4bd1-80ea-479570f294ff None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:25.306501 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7517ef-e4a2-4e51-a50c-63460b134daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:25.307160 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7517ef-e4a2-4e51-a50c-63460b134daf admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:25.307463 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7517ef-e4a2-4e51-a50c-63460b134daf admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:25.307733 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7517ef-e4a2-4e51-a50c-63460b134daf admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:25.308433 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7517ef-e4a2-4e51-a50c-63460b134daf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:33:25.308765 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7517ef-e4a2-4e51-a50c-63460b134daf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:25.310373 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7517ef-e4a2-4e51-a50c-63460b134daf admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:25.322776 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1122/2242] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:33:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:33:25.327674 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fabf54-fb4d-47cc-9987-92142a8d32ca None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:25.337211 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fabf54-fb4d-47cc-9987-92142a8d32ca None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:25.342435 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a80f03-61e5-4a61-a44a-5eb832660579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:25.343172 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a80f03-61e5-4a61-a44a-5eb832660579 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:25.343477 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a80f03-61e5-4a61-a44a-5eb832660579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:25.343761 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a80f03-61e5-4a61-a44a-5eb832660579 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:25.344423 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a80f03-61e5-4a61-a44a-5eb832660579 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:25.346354 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a80f03-61e5-4a61-a44a-5eb832660579 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:25.376662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1121/2243] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:33:25 2022] POST /identity/v3/projects => generated 363 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:33:25.381038 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7517ef-e4a2-4e51-a50c-63460b134daf admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:25.391777 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7517ef-e4a2-4e51-a50c-63460b134daf admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:25.418187 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a80f03-61e5-4a61-a44a-5eb832660579 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:25.420034 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-60a80f03-61e5-4a61-a44a-5eb832660579 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:25.449518 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306f42c8-c3e4-4aa6-bb80-f59812461381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:25.450785 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306f42c8-c3e4-4aa6-bb80-f59812461381 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:25.451103 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306f42c8-c3e4-4aa6-bb80-f59812461381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:25.451381 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306f42c8-c3e4-4aa6-bb80-f59812461381 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:25.452107 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306f42c8-c3e4-4aa6-bb80-f59812461381 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:25.454101 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306f42c8-c3e4-4aa6-bb80-f59812461381 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:25.477096 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1123/2244] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:33:25 2022] POST /identity/v3/users => generated 282 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:33:25.480369 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a80f03-61e5-4a61-a44a-5eb832660579 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:25.481215 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306f42c8-c3e4-4aa6-bb80-f59812461381 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:25.484281 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590df94f-12e7-428a-b633-9404bb255845 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:25.484908 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590df94f-12e7-428a-b633-9404bb255845 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:25.485187 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590df94f-12e7-428a-b633-9404bb255845 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:25.485459 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590df94f-12e7-428a-b633-9404bb255845 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:25.487920 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-590df94f-12e7-428a-b633-9404bb255845 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:25.491020 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306f42c8-c3e4-4aa6-bb80-f59812461381 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:25.495792 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc09f8-46cb-46ef-9378-1ebaaa0c6097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:25.496483 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc09f8-46cb-46ef-9378-1ebaaa0c6097 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:25.496774 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc09f8-46cb-46ef-9378-1ebaaa0c6097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:25.497052 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc09f8-46cb-46ef-9378-1ebaaa0c6097 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:25.497763 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590df94f-12e7-428a-b633-9404bb255845 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:25.497863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afc09f8-46cb-46ef-9378-1ebaaa0c6097 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:25.499565 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590df94f-12e7-428a-b633-9404bb255845 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:25.499675 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afc09f8-46cb-46ef-9378-1ebaaa0c6097 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:25.507871 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1122/2245] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:33:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 10:33:25.513117 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1124/2246] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:33:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 10:33:25.514628 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590df94f-12e7-428a-b633-9404bb255845 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:25.526392 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590df94f-12e7-428a-b633-9404bb255845 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:25.531207 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b418848-9247-491c-bdf0-6cf48106c170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:25.531854 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b418848-9247-491c-bdf0-6cf48106c170 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:25.532136 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b418848-9247-491c-bdf0-6cf48106c170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:25.532410 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b418848-9247-491c-bdf0-6cf48106c170 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:25.533105 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b418848-9247-491c-bdf0-6cf48106c170 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:25.534715 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b418848-9247-491c-bdf0-6cf48106c170 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:25.545247 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1123/2247] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:33:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:33:25.549546 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc09f8-46cb-46ef-9378-1ebaaa0c6097 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:25.560299 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc09f8-46cb-46ef-9378-1ebaaa0c6097 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:25.565007 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c60f485-84ca-4ff9-b4be-6a1a6e41a171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:25.565782 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c60f485-84ca-4ff9-b4be-6a1a6e41a171 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:25.566091 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c60f485-84ca-4ff9-b4be-6a1a6e41a171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:25.566364 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c60f485-84ca-4ff9-b4be-6a1a6e41a171 admin admin] PATH_INFO: `/v3/projects/4764c450e15e42d08a89ba38d9bcd96a/users/8f34d797c213461592b84b0c50250192/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:25.581144 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c60f485-84ca-4ff9-b4be-6a1a6e41a171 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4764c450e15e42d08a89ba38d9bcd96a, user_id=8f34d797c213461592b84b0c50250192, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:25.583877 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c60f485-84ca-4ff9-b4be-6a1a6e41a171 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:25.598955 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1125/2248] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:33:25 2022] PUT /identity/v3/projects/4764c450e15e42d08a89ba38d9bcd96a/users/8f34d797c213461592b84b0c50250192/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:33:25.610532 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac5792-8874-4f63-b0e4-934db26ff1cf None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:25.610822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac5792-8874-4f63-b0e4-934db26ff1cf None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:25.611137 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac5792-8874-4f63-b0e4-934db26ff1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:25.641007 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-72ac5792-8874-4f63-b0e4-934db26ff1cf None None] MFA Rules not processed for user `8f34d797c213461592b84b0c50250192`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:33:25.642238 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-72ac5792-8874-4f63-b0e4-934db26ff1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:25.673920 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:33:25.714956 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1124/2249] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:33:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:33:25.719969 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c60f485-84ca-4ff9-b4be-6a1a6e41a171 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:25.765393 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:33:25.792332 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c60f485-84ca-4ff9-b4be-6a1a6e41a171 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:25.797121 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbb66bb-bacc-42cd-b263-971956b77904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:25.797747 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb66bb-bacc-42cd-b263-971956b77904 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:25.798029 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb66bb-bacc-42cd-b263-971956b77904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:25.798289 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb66bb-bacc-42cd-b263-971956b77904 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:25.798979 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbb66bb-bacc-42cd-b263-971956b77904 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:25.800885 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbb66bb-bacc-42cd-b263-971956b77904 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:25.858330 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1126/2250] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:33:25 2022] POST /identity/v3/projects => generated 363 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:33:25.862580 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ac5792-8874-4f63-b0e4-934db26ff1cf None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:25.874218 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ac5792-8874-4f63-b0e4-934db26ff1cf None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:25.946225 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46299c83-8950-4fa5-b0e7-8fb4c94a004c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:25.946890 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46299c83-8950-4fa5-b0e7-8fb4c94a004c admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:25.947201 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46299c83-8950-4fa5-b0e7-8fb4c94a004c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:25.947477 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46299c83-8950-4fa5-b0e7-8fb4c94a004c admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:25.948166 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46299c83-8950-4fa5-b0e7-8fb4c94a004c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:25.950518 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46299c83-8950-4fa5-b0e7-8fb4c94a004c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:25.972694 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1125/2251] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:33:25 2022] POST /identity/v3/users => generated 288 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:33:25.976814 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbb66bb-bacc-42cd-b263-971956b77904 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:25.987452 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbb66bb-bacc-42cd-b263-971956b77904 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:25.992643 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cabbfd-35fb-40b8-a964-3d14cba3ca94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:25.993284 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cabbfd-35fb-40b8-a964-3d14cba3ca94 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:25.993582 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cabbfd-35fb-40b8-a964-3d14cba3ca94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:25.993850 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cabbfd-35fb-40b8-a964-3d14cba3ca94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:25.994538 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cabbfd-35fb-40b8-a964-3d14cba3ca94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:25.996156 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cabbfd-35fb-40b8-a964-3d14cba3ca94 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:26.007099 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1127/2252] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:33:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:33:26.011420 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46299c83-8950-4fa5-b0e7-8fb4c94a004c admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:26.021666 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46299c83-8950-4fa5-b0e7-8fb4c94a004c admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:26.027439 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774e4cd9-efcb-41d1-aed5-7a31ba135694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:26.028133 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e4cd9-efcb-41d1-aed5-7a31ba135694 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:26.028455 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e4cd9-efcb-41d1-aed5-7a31ba135694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:26.028778 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e4cd9-efcb-41d1-aed5-7a31ba135694 admin admin] PATH_INFO: `/v3/projects/a2faa5ff8f614e87b894fa843951fd23/users/8c28dafc813b4df3b5ffd0f8c726284a/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:26.052455 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774e4cd9-efcb-41d1-aed5-7a31ba135694 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2faa5ff8f614e87b894fa843951fd23, user_id=8c28dafc813b4df3b5ffd0f8c726284a, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:26.055179 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774e4cd9-efcb-41d1-aed5-7a31ba135694 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:26.069849 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1126/2253] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:33:26 2022] PUT /identity/v3/projects/a2faa5ff8f614e87b894fa843951fd23/users/8c28dafc813b4df3b5ffd0f8c726284a/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:33:26.073757 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cabbfd-35fb-40b8-a964-3d14cba3ca94 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:26.084049 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cabbfd-35fb-40b8-a964-3d14cba3ca94 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:26.141359 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f014a29a-2763-4d1c-af7c-448eb5db01cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:26.141988 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f014a29a-2763-4d1c-af7c-448eb5db01cb admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:26.142261 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f014a29a-2763-4d1c-af7c-448eb5db01cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:26.142861 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f014a29a-2763-4d1c-af7c-448eb5db01cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:26.143252 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f014a29a-2763-4d1c-af7c-448eb5db01cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:26.144849 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f014a29a-2763-4d1c-af7c-448eb5db01cb admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:26.158729 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1128/2254] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:33:26 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:33:26.161064 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774e4cd9-efcb-41d1-aed5-7a31ba135694 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:26.173202 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774e4cd9-efcb-41d1-aed5-7a31ba135694 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:26.177794 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f890b51b-cd7a-41e2-bcea-4876a7df7abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:26.178468 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890b51b-cd7a-41e2-bcea-4876a7df7abe admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:26.178767 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890b51b-cd7a-41e2-bcea-4876a7df7abe admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:26.179157 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890b51b-cd7a-41e2-bcea-4876a7df7abe admin admin] PATH_INFO: `/v3/projects/a2faa5ff8f614e87b894fa843951fd23/users/8c28dafc813b4df3b5ffd0f8c726284a/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:26.181413 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f890b51b-cd7a-41e2-bcea-4876a7df7abe admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2faa5ff8f614e87b894fa843951fd23, user_id=8c28dafc813b4df3b5ffd0f8c726284a, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:26.184168 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f890b51b-cd7a-41e2-bcea-4876a7df7abe admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:26.196042 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1127/2255] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:33:26 2022] PUT /identity/v3/projects/a2faa5ff8f614e87b894fa843951fd23/users/8c28dafc813b4df3b5ffd0f8c726284a/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:33:26.207313 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a45c4c5-2808-4948-82ec-a3d57b0dd19b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:26.207597 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a45c4c5-2808-4948-82ec-a3d57b0dd19b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:26.207986 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a45c4c5-2808-4948-82ec-a3d57b0dd19b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:26.236812 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8a45c4c5-2808-4948-82ec-a3d57b0dd19b None None] MFA Rules not processed for user `8c28dafc813b4df3b5ffd0f8c726284a`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:33:26.238032 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8a45c4c5-2808-4948-82ec-a3d57b0dd19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:26.314197 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1129/2256] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:33:26 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:33:26.326111 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f890b51b-cd7a-41e2-bcea-4876a7df7abe admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:26.328117 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f890b51b-cd7a-41e2-bcea-4876a7df7abe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:26.388203 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f890b51b-cd7a-41e2-bcea-4876a7df7abe admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:26.392691 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fbb413-7082-4124-8c57-74c70b5db92d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:26.393309 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fbb413-7082-4124-8c57-74c70b5db92d service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:26.393592 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fbb413-7082-4124-8c57-74c70b5db92d service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:26.393863 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fbb413-7082-4124-8c57-74c70b5db92d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:26.403800 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fbb413-7082-4124-8c57-74c70b5db92d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:26.405926 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fbb413-7082-4124-8c57-74c70b5db92d service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:26.415903 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1128/2257] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:33:26 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:33:26.499112 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a45c4c5-2808-4948-82ec-a3d57b0dd19b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:26.509455 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a45c4c5-2808-4948-82ec-a3d57b0dd19b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:26.513849 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff6ff41-dcfd-49ff-934b-776be209bd44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:26.514504 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6ff41-dcfd-49ff-934b-776be209bd44 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:26.514790 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6ff41-dcfd-49ff-934b-776be209bd44 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:26.515092 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6ff41-dcfd-49ff-934b-776be209bd44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:26.547697 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:33:26.574814 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff6ff41-dcfd-49ff-934b-776be209bd44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:26.576251 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff6ff41-dcfd-49ff-934b-776be209bd44 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:26.585209 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1130/2258] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:33:26 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:33:29.479867 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fbb413-7082-4124-8c57-74c70b5db92d service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:29.509256 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:33:29.551758 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fbb413-7082-4124-8c57-74c70b5db92d service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:29.556798 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116924c1-e3e4-4f32-a67c-a52b61e29c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:29.557453 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116924c1-e3e4-4f32-a67c-a52b61e29c5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:29.557728 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116924c1-e3e4-4f32-a67c-a52b61e29c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:29.557995 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116924c1-e3e4-4f32-a67c-a52b61e29c5e admin admin] PATH_INFO: `/v3/users/8f34d797c213461592b84b0c50250192` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:29.559289 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116924c1-e3e4-4f32-a67c-a52b61e29c5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f34d797c213461592b84b0c50250192)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:29.561207 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116924c1-e3e4-4f32-a67c-a52b61e29c5e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:29.615687 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-116924c1-e3e4-4f32-a67c-a52b61e29c5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f34d797c213461592b84b0c50250192'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:29.625030 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-116924c1-e3e4-4f32-a67c-a52b61e29c5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f34d797c213461592b84b0c50250192'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:29.625688 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-116924c1-e3e4-4f32-a67c-a52b61e29c5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f34d797c213461592b84b0c50250192'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:29.637267 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-116924c1-e3e4-4f32-a67c-a52b61e29c5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f34d797c213461592b84b0c50250192'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:29.638101 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-116924c1-e3e4-4f32-a67c-a52b61e29c5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f34d797c213461592b84b0c50250192'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:29.650387 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1129/2259] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:33:29 2022] DELETE /identity/v3/users/8f34d797c213461592b84b0c50250192 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:33:29.655133 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff727741-5a42-4f5d-a586-89e8c0d29a87 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:29.655620 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff727741-5a42-4f5d-a586-89e8c0d29a87 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:29.655735 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff727741-5a42-4f5d-a586-89e8c0d29a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:29.675211 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ff727741-5a42-4f5d-a586-89e8c0d29a87 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:33:29.676537 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ff727741-5a42-4f5d-a586-89e8c0d29a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:29.741084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1131/2260] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:33:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:33:29.751344 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116924c1-e3e4-4f32-a67c-a52b61e29c5e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:29.752990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-116924c1-e3e4-4f32-a67c-a52b61e29c5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:29.820925 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116924c1-e3e4-4f32-a67c-a52b61e29c5e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:29.827281 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8175a24a-3b6a-486c-aea5-9becd175a8ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:29.827910 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175a24a-3b6a-486c-aea5-9becd175a8ee service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:29.828180 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175a24a-3b6a-486c-aea5-9becd175a8ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:29.828465 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175a24a-3b6a-486c-aea5-9becd175a8ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:29.838045 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8175a24a-3b6a-486c-aea5-9becd175a8ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:29.839521 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8175a24a-3b6a-486c-aea5-9becd175a8ee service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:29.849107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1130/2261] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:33:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:33:30.130348 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff727741-5a42-4f5d-a586-89e8c0d29a87 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:30.131828 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ff727741-5a42-4f5d-a586-89e8c0d29a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:30.145026 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff727741-5a42-4f5d-a586-89e8c0d29a87 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:30.150923 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64a88d-bdaa-4709-999e-20724bb9c221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:30.151708 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64a88d-bdaa-4709-999e-20724bb9c221 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:30.152077 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64a88d-bdaa-4709-999e-20724bb9c221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:30.152356 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64a88d-bdaa-4709-999e-20724bb9c221 admin admin] PATH_INFO: `/v3/projects/4764c450e15e42d08a89ba38d9bcd96a` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:30.153657 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a64a88d-bdaa-4709-999e-20724bb9c221 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4764c450e15e42d08a89ba38d9bcd96a)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:30.156261 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a64a88d-bdaa-4709-999e-20724bb9c221 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:30.207488 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0a64a88d-bdaa-4709-999e-20724bb9c221 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4764c450e15e42d08a89ba38d9bcd96a', 'request_id': 'req-0a64a88d-bdaa-4709-999e-20724bb9c221'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:30.218000 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0a64a88d-bdaa-4709-999e-20724bb9c221 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4764c450e15e42d08a89ba38d9bcd96a', 'request_id': 'req-0a64a88d-bdaa-4709-999e-20724bb9c221'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:30.224487 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0a64a88d-bdaa-4709-999e-20724bb9c221 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4764c450e15e42d08a89ba38d9bcd96a', 'request_id': 'req-0a64a88d-bdaa-4709-999e-20724bb9c221'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:30.235990 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0a64a88d-bdaa-4709-999e-20724bb9c221 admin admin] The token cache is being invalidate because project 4764c450e15e42d08a89ba38d9bcd96a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:33:30.236259 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0a64a88d-bdaa-4709-999e-20724bb9c221 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:30.236881 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0a64a88d-bdaa-4709-999e-20724bb9c221 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:30.238410 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1132/2262] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:33:30 2022] DELETE /identity/v3/projects/4764c450e15e42d08a89ba38d9bcd96a => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:33:30.242459 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8175a24a-3b6a-486c-aea5-9becd175a8ee service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:30.252480 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8175a24a-3b6a-486c-aea5-9becd175a8ee service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:30.300467 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:33:30.320519 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef35b72f-d227-458e-a1f8-f9bf232845a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:30.321389 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35b72f-d227-458e-a1f8-f9bf232845a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:30.321768 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35b72f-d227-458e-a1f8-f9bf232845a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:30.322104 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35b72f-d227-458e-a1f8-f9bf232845a3 admin admin] PATH_INFO: `/v3/users/8c28dafc813b4df3b5ffd0f8c726284a` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:30.324003 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef35b72f-d227-458e-a1f8-f9bf232845a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c28dafc813b4df3b5ffd0f8c726284a)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:30.327041 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef35b72f-d227-458e-a1f8-f9bf232845a3 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:30.377558 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ef35b72f-d227-458e-a1f8-f9bf232845a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c28dafc813b4df3b5ffd0f8c726284a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:30.386341 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ef35b72f-d227-458e-a1f8-f9bf232845a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c28dafc813b4df3b5ffd0f8c726284a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:30.387082 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ef35b72f-d227-458e-a1f8-f9bf232845a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c28dafc813b4df3b5ffd0f8c726284a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:30.401223 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ef35b72f-d227-458e-a1f8-f9bf232845a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c28dafc813b4df3b5ffd0f8c726284a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:30.401875 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ef35b72f-d227-458e-a1f8-f9bf232845a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c28dafc813b4df3b5ffd0f8c726284a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:30.414298 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1131/2263] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:33:30 2022] DELETE /identity/v3/users/8c28dafc813b4df3b5ffd0f8c726284a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:33:30.766829 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64a88d-bdaa-4709-999e-20724bb9c221 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:30.768911 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0a64a88d-bdaa-4709-999e-20724bb9c221 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:30.786172 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64a88d-bdaa-4709-999e-20724bb9c221 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:30.839303 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:33:30.847295 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dfe4f4-9bf0-4c3a-820c-c3022edc0a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:30.847984 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dfe4f4-9bf0-4c3a-820c-c3022edc0a98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:30.848461 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dfe4f4-9bf0-4c3a-820c-c3022edc0a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:30.849033 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dfe4f4-9bf0-4c3a-820c-c3022edc0a98 admin admin] PATH_INFO: `/v3/projects/a2faa5ff8f614e87b894fa843951fd23` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:30.850474 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dfe4f4-9bf0-4c3a-820c-c3022edc0a98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2faa5ff8f614e87b894fa843951fd23)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:30.852906 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dfe4f4-9bf0-4c3a-820c-c3022edc0a98 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:30.903857 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-20dfe4f4-9bf0-4c3a-820c-c3022edc0a98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2faa5ff8f614e87b894fa843951fd23', 'request_id': 'req-20dfe4f4-9bf0-4c3a-820c-c3022edc0a98'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:30.915264 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-20dfe4f4-9bf0-4c3a-820c-c3022edc0a98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2faa5ff8f614e87b894fa843951fd23', 'request_id': 'req-20dfe4f4-9bf0-4c3a-820c-c3022edc0a98'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:30.921715 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-20dfe4f4-9bf0-4c3a-820c-c3022edc0a98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2faa5ff8f614e87b894fa843951fd23', 'request_id': 'req-20dfe4f4-9bf0-4c3a-820c-c3022edc0a98'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:30.933634 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-20dfe4f4-9bf0-4c3a-820c-c3022edc0a98 admin admin] The token cache is being invalidate because project a2faa5ff8f614e87b894fa843951fd23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:33:30.933960 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-20dfe4f4-9bf0-4c3a-820c-c3022edc0a98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:30.934680 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-20dfe4f4-9bf0-4c3a-820c-c3022edc0a98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:30.936322 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1133/2264] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:33:30 2022] DELETE /identity/v3/projects/a2faa5ff8f614e87b894fa843951fd23 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:33:30.941678 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90747b10-9187-45c0-8a77-91218b2c7be1 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:30.942507 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90747b10-9187-45c0-8a77-91218b2c7be1 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:30.943050 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90747b10-9187-45c0-8a77-91218b2c7be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:30.963539 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-90747b10-9187-45c0-8a77-91218b2c7be1 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:33:30.964761 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-90747b10-9187-45c0-8a77-91218b2c7be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:31.026188 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1132/2265] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:33:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:33:31.034679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4be30d-b6d5-4875-89ad-95cb94821880 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:31.034980 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4be30d-b6d5-4875-89ad-95cb94821880 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:31.035256 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4be30d-b6d5-4875-89ad-95cb94821880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:31.056439 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8c4be30d-b6d5-4875-89ad-95cb94821880 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:33:31.057681 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8c4be30d-b6d5-4875-89ad-95cb94821880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:31.066604 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1134/2266] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:33:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:33:31.072386 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90747b10-9187-45c0-8a77-91218b2c7be1 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:31.084289 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90747b10-9187-45c0-8a77-91218b2c7be1 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:31.089171 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1040e15b-6915-4ad9-861d-36be6e67ca1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:31.089789 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1040e15b-6915-4ad9-861d-36be6e67ca1c admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:31.090068 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1040e15b-6915-4ad9-861d-36be6e67ca1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:31.090341 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1040e15b-6915-4ad9-861d-36be6e67ca1c admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:31.091097 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1040e15b-6915-4ad9-861d-36be6e67ca1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:33:31.091402 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1040e15b-6915-4ad9-861d-36be6e67ca1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:31.093030 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1040e15b-6915-4ad9-861d-36be6e67ca1c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:31.104889 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1133/2267] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:33:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:33:31.109681 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4be30d-b6d5-4875-89ad-95cb94821880 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:31.124035 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4be30d-b6d5-4875-89ad-95cb94821880 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:31.131592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd340afe-0db5-41d4-ae3a-dc1b2cb1e9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:31.132478 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd340afe-0db5-41d4-ae3a-dc1b2cb1e9cc admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:31.132857 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd340afe-0db5-41d4-ae3a-dc1b2cb1e9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:31.133272 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd340afe-0db5-41d4-ae3a-dc1b2cb1e9cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:31.134248 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd340afe-0db5-41d4-ae3a-dc1b2cb1e9cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:31.137167 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd340afe-0db5-41d4-ae3a-dc1b2cb1e9cc admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:31.169796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1135/2268] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:33:31 2022] POST /identity/v3/projects => generated 367 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:33:31.174852 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1040e15b-6915-4ad9-861d-36be6e67ca1c admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:31.185386 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1040e15b-6915-4ad9-861d-36be6e67ca1c admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:31.233632 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:33:31.246439 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b7aea-37d5-47b7-ae48-5b5eedac997f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:31.247623 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b7aea-37d5-47b7-ae48-5b5eedac997f admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:31.247978 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b7aea-37d5-47b7-ae48-5b5eedac997f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:31.248353 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b7aea-37d5-47b7-ae48-5b5eedac997f admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:31.249387 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b7aea-37d5-47b7-ae48-5b5eedac997f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:31.251981 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b7aea-37d5-47b7-ae48-5b5eedac997f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:31.275024 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1134/2269] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:33:31 2022] POST /identity/v3/users => generated 284 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:33:31.279515 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd340afe-0db5-41d4-ae3a-dc1b2cb1e9cc admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:31.290636 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd340afe-0db5-41d4-ae3a-dc1b2cb1e9cc admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:31.295357 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a6dc0-5993-4c2a-ad1b-2ce76dac2486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:31.295969 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a6dc0-5993-4c2a-ad1b-2ce76dac2486 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:31.296244 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a6dc0-5993-4c2a-ad1b-2ce76dac2486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:31.296684 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a6dc0-5993-4c2a-ad1b-2ce76dac2486 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:31.297416 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72a6dc0-5993-4c2a-ad1b-2ce76dac2486 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:31.299046 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72a6dc0-5993-4c2a-ad1b-2ce76dac2486 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:31.309576 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1136/2270] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:33:31.313798 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b7aea-37d5-47b7-ae48-5b5eedac997f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:31.323650 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b7aea-37d5-47b7-ae48-5b5eedac997f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:31.328349 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe260482-3d1c-47b4-a99c-c59565b77454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:31.328953 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe260482-3d1c-47b4-a99c-c59565b77454 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:31.329231 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe260482-3d1c-47b4-a99c-c59565b77454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:31.329492 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe260482-3d1c-47b4-a99c-c59565b77454 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:31.330179 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe260482-3d1c-47b4-a99c-c59565b77454 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:31.331813 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe260482-3d1c-47b4-a99c-c59565b77454 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:31.342390 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1135/2271] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:33:31.346773 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a6dc0-5993-4c2a-ad1b-2ce76dac2486 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:31.357448 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a6dc0-5993-4c2a-ad1b-2ce76dac2486 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:31.362840 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51d091-d8c4-4e7b-af4b-46185a9cecd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:31.363536 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51d091-d8c4-4e7b-af4b-46185a9cecd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:31.363811 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51d091-d8c4-4e7b-af4b-46185a9cecd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:31.364257 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51d091-d8c4-4e7b-af4b-46185a9cecd1 admin admin] PATH_INFO: `/v3/projects/f52d88f46a044e0990353a4a5e311985/users/8c1e9a659b8f4d24a6aea5f45e51bd36/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:31.380225 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c51d091-d8c4-4e7b-af4b-46185a9cecd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f52d88f46a044e0990353a4a5e311985, user_id=8c1e9a659b8f4d24a6aea5f45e51bd36, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:31.383059 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c51d091-d8c4-4e7b-af4b-46185a9cecd1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:31.397441 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1137/2272] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:33:31 2022] PUT /identity/v3/projects/f52d88f46a044e0990353a4a5e311985/users/8c1e9a659b8f4d24a6aea5f45e51bd36/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:33:31.409755 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76168451-5168-4f64-853b-48e9807a38c2 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:31.410147 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76168451-5168-4f64-853b-48e9807a38c2 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:31.410433 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76168451-5168-4f64-853b-48e9807a38c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:31.440467 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-76168451-5168-4f64-853b-48e9807a38c2 None None] MFA Rules not processed for user `8c1e9a659b8f4d24a6aea5f45e51bd36`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:33:31.441688 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-76168451-5168-4f64-853b-48e9807a38c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:31.509279 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1136/2273] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:33:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:33:31.514098 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51d091-d8c4-4e7b-af4b-46185a9cecd1 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:31.576817 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51d091-d8c4-4e7b-af4b-46185a9cecd1 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:31.581794 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6558b-129f-4863-b712-b3a02d53525f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:31.582413 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6558b-129f-4863-b712-b3a02d53525f admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:31.582689 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6558b-129f-4863-b712-b3a02d53525f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:31.582966 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6558b-129f-4863-b712-b3a02d53525f admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:31.583651 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a6558b-129f-4863-b712-b3a02d53525f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:31.585551 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a6558b-129f-4863-b712-b3a02d53525f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:31.618409 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1138/2274] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:33:31 2022] POST /identity/v3/projects => generated 367 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:33:31.623461 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76168451-5168-4f64-853b-48e9807a38c2 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:31.635212 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76168451-5168-4f64-853b-48e9807a38c2 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:31.696085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67918a31-0def-4f78-8e16-970a5d5a591d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:31.696713 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67918a31-0def-4f78-8e16-970a5d5a591d admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:31.696996 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67918a31-0def-4f78-8e16-970a5d5a591d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:31.697267 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67918a31-0def-4f78-8e16-970a5d5a591d admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:31.697938 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67918a31-0def-4f78-8e16-970a5d5a591d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:31.699883 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67918a31-0def-4f78-8e16-970a5d5a591d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:31.728465 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1137/2275] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:33:31 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:33:31.729727 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6558b-129f-4863-b712-b3a02d53525f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:31.740433 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6558b-129f-4863-b712-b3a02d53525f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:31.745249 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657fdd74-c426-42ea-b09a-c26a3d125171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:31.745941 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657fdd74-c426-42ea-b09a-c26a3d125171 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:31.746216 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657fdd74-c426-42ea-b09a-c26a3d125171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:31.746480 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657fdd74-c426-42ea-b09a-c26a3d125171 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:31.747200 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657fdd74-c426-42ea-b09a-c26a3d125171 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:31.748822 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657fdd74-c426-42ea-b09a-c26a3d125171 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:31.750426 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:33:31.764418 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1139/2276] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:33:31.765483 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67918a31-0def-4f78-8e16-970a5d5a591d admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:31.776478 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67918a31-0def-4f78-8e16-970a5d5a591d admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:31.782271 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69713ba2-7c06-4e03-9286-311ae70c07df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:31.783085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69713ba2-7c06-4e03-9286-311ae70c07df admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:31.783361 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69713ba2-7c06-4e03-9286-311ae70c07df admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:31.783685 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69713ba2-7c06-4e03-9286-311ae70c07df admin admin] PATH_INFO: `/v3/projects/0c89c41c1ccb4ae89338efe6e5470489/users/52c797736d6b4d13ad5d661801f3b4ff/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:31.800376 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69713ba2-7c06-4e03-9286-311ae70c07df admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c89c41c1ccb4ae89338efe6e5470489, user_id=52c797736d6b4d13ad5d661801f3b4ff, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:31.803088 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69713ba2-7c06-4e03-9286-311ae70c07df admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:31.818478 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1138/2277] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:33:31 2022] PUT /identity/v3/projects/0c89c41c1ccb4ae89338efe6e5470489/users/52c797736d6b4d13ad5d661801f3b4ff/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:33:31.822702 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657fdd74-c426-42ea-b09a-c26a3d125171 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:31.833490 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657fdd74-c426-42ea-b09a-c26a3d125171 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:31.901907 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c9613-7ce2-4c2e-a188-1e92eee4b4f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:31.902561 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c9613-7ce2-4c2e-a188-1e92eee4b4f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:31.902838 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c9613-7ce2-4c2e-a188-1e92eee4b4f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:31.903139 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c9613-7ce2-4c2e-a188-1e92eee4b4f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:31.903828 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623c9613-7ce2-4c2e-a188-1e92eee4b4f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:31.905465 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623c9613-7ce2-4c2e-a188-1e92eee4b4f7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:31.915538 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1140/2278] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:33:31.919688 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69713ba2-7c06-4e03-9286-311ae70c07df admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:31.931405 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69713ba2-7c06-4e03-9286-311ae70c07df admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:31.937539 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eea719-74f8-43ee-af94-d2397db0a2cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:31.938258 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eea719-74f8-43ee-af94-d2397db0a2cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:31.938536 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eea719-74f8-43ee-af94-d2397db0a2cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:31.938806 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eea719-74f8-43ee-af94-d2397db0a2cd admin admin] PATH_INFO: `/v3/projects/0c89c41c1ccb4ae89338efe6e5470489/users/52c797736d6b4d13ad5d661801f3b4ff/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:31.940877 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eea719-74f8-43ee-af94-d2397db0a2cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c89c41c1ccb4ae89338efe6e5470489, user_id=52c797736d6b4d13ad5d661801f3b4ff, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:31.943524 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eea719-74f8-43ee-af94-d2397db0a2cd admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:31.955818 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1139/2279] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:33:31 2022] PUT /identity/v3/projects/0c89c41c1ccb4ae89338efe6e5470489/users/52c797736d6b4d13ad5d661801f3b4ff/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:33:31.966722 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0615435d-cd25-4655-aeb7-66e94bf5cb0d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:31.967036 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0615435d-cd25-4655-aeb7-66e94bf5cb0d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:31.967322 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0615435d-cd25-4655-aeb7-66e94bf5cb0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:31.997379 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-0615435d-cd25-4655-aeb7-66e94bf5cb0d None None] MFA Rules not processed for user `52c797736d6b4d13ad5d661801f3b4ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:33:31.998586 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0615435d-cd25-4655-aeb7-66e94bf5cb0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:32.066385 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1141/2280] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:33:31 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:33:32.078150 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eea719-74f8-43ee-af94-d2397db0a2cd admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:32.080060 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-16eea719-74f8-43ee-af94-d2397db0a2cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:32.095259 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:33:32.141968 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eea719-74f8-43ee-af94-d2397db0a2cd admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:32.145909 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b426c8-8faf-4eb4-ada7-7ea17a76b36d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:32.146526 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b426c8-8faf-4eb4-ada7-7ea17a76b36d service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:32.146801 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b426c8-8faf-4eb4-ada7-7ea17a76b36d service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:32.147097 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b426c8-8faf-4eb4-ada7-7ea17a76b36d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:32.157414 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b426c8-8faf-4eb4-ada7-7ea17a76b36d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:32.158870 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b426c8-8faf-4eb4-ada7-7ea17a76b36d service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:32.172279 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1140/2281] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:33:32 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:33:32.219817 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0615435d-cd25-4655-aeb7-66e94bf5cb0d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:32.221450 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0615435d-cd25-4655-aeb7-66e94bf5cb0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:32.233802 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0615435d-cd25-4655-aeb7-66e94bf5cb0d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:32.238510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851467b1-c6d1-4211-91b1-cf69287f0722 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:32.239147 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851467b1-c6d1-4211-91b1-cf69287f0722 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:32.239424 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851467b1-c6d1-4211-91b1-cf69287f0722 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:32.239697 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851467b1-c6d1-4211-91b1-cf69287f0722 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:32.301317 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851467b1-c6d1-4211-91b1-cf69287f0722 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:32.302792 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851467b1-c6d1-4211-91b1-cf69287f0722 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:32.313189 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1142/2282] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:33:32 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:33:35.724688 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b426c8-8faf-4eb4-ada7-7ea17a76b36d service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:35.790648 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b426c8-8faf-4eb4-ada7-7ea17a76b36d service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:35.795065 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde24955-876c-427e-a274-63c30cef8e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:35.795709 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde24955-876c-427e-a274-63c30cef8e4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:35.796017 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde24955-876c-427e-a274-63c30cef8e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:35.796374 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde24955-876c-427e-a274-63c30cef8e4e admin admin] PATH_INFO: `/v3/users/8c1e9a659b8f4d24a6aea5f45e51bd36` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:35.797527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde24955-876c-427e-a274-63c30cef8e4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c1e9a659b8f4d24a6aea5f45e51bd36)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:35.799480 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde24955-876c-427e-a274-63c30cef8e4e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:35.847000 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-bde24955-876c-427e-a274-63c30cef8e4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c1e9a659b8f4d24a6aea5f45e51bd36'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:35.856941 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-bde24955-876c-427e-a274-63c30cef8e4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c1e9a659b8f4d24a6aea5f45e51bd36'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:35.857563 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-bde24955-876c-427e-a274-63c30cef8e4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c1e9a659b8f4d24a6aea5f45e51bd36'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:35.869231 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-bde24955-876c-427e-a274-63c30cef8e4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c1e9a659b8f4d24a6aea5f45e51bd36'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:35.869860 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-bde24955-876c-427e-a274-63c30cef8e4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c1e9a659b8f4d24a6aea5f45e51bd36'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:35.882976 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1141/2283] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:33:35 2022] DELETE /identity/v3/users/8c1e9a659b8f4d24a6aea5f45e51bd36 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:33:35.887781 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd000338-13ee-4beb-9e32-187c645a0887 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:35.888293 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd000338-13ee-4beb-9e32-187c645a0887 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:35.888395 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd000338-13ee-4beb-9e32-187c645a0887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:35.908833 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-cd000338-13ee-4beb-9e32-187c645a0887 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:33:35.910032 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cd000338-13ee-4beb-9e32-187c645a0887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:35.919775 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:33:35.978692 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1143/2284] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:33:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:33:35.988772 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde24955-876c-427e-a274-63c30cef8e4e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:35.990342 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-bde24955-876c-427e-a274-63c30cef8e4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:36.034358 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:33:36.055202 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde24955-876c-427e-a274-63c30cef8e4e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:36.059563 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ecb685-4e23-45ec-967e-c57b8ae231e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:36.060187 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecb685-4e23-45ec-967e-c57b8ae231e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:36.060462 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecb685-4e23-45ec-967e-c57b8ae231e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:36.060729 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecb685-4e23-45ec-967e-c57b8ae231e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:36.070930 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ecb685-4e23-45ec-967e-c57b8ae231e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:36.072475 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ecb685-4e23-45ec-967e-c57b8ae231e6 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:36.082438 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1142/2285] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:33:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:33:36.364519 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd000338-13ee-4beb-9e32-187c645a0887 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:36.365996 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cd000338-13ee-4beb-9e32-187c645a0887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:36.379800 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd000338-13ee-4beb-9e32-187c645a0887 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:36.384632 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fb5e41-5c14-43a7-8559-c8744b452fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:36.385281 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb5e41-5c14-43a7-8559-c8744b452fcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:36.385992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb5e41-5c14-43a7-8559-c8744b452fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:36.385992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb5e41-5c14-43a7-8559-c8744b452fcf admin admin] PATH_INFO: `/v3/projects/f52d88f46a044e0990353a4a5e311985` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:36.386988 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fb5e41-5c14-43a7-8559-c8744b452fcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=f52d88f46a044e0990353a4a5e311985)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:36.388995 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fb5e41-5c14-43a7-8559-c8744b452fcf admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:36.441344 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-64fb5e41-5c14-43a7-8559-c8744b452fcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f52d88f46a044e0990353a4a5e311985', 'request_id': 'req-64fb5e41-5c14-43a7-8559-c8744b452fcf'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:36.450204 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-64fb5e41-5c14-43a7-8559-c8744b452fcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f52d88f46a044e0990353a4a5e311985', 'request_id': 'req-64fb5e41-5c14-43a7-8559-c8744b452fcf'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:36.457248 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-64fb5e41-5c14-43a7-8559-c8744b452fcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f52d88f46a044e0990353a4a5e311985', 'request_id': 'req-64fb5e41-5c14-43a7-8559-c8744b452fcf'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:36.468529 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-64fb5e41-5c14-43a7-8559-c8744b452fcf admin admin] The token cache is being invalidate because project f52d88f46a044e0990353a4a5e311985 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:33:36.468529 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-64fb5e41-5c14-43a7-8559-c8744b452fcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:36.469524 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-64fb5e41-5c14-43a7-8559-c8744b452fcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:36.471085 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1144/2286] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:33:36 2022] DELETE /identity/v3/projects/f52d88f46a044e0990353a4a5e311985 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:33:36.475139 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ecb685-4e23-45ec-967e-c57b8ae231e6 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:36.485462 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ecb685-4e23-45ec-967e-c57b8ae231e6 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:36.545280 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f02f0-2840-4556-8a5b-567d485f04cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:36.545950 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f02f0-2840-4556-8a5b-567d485f04cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:36.546249 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f02f0-2840-4556-8a5b-567d485f04cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:36.546587 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f02f0-2840-4556-8a5b-567d485f04cd admin admin] PATH_INFO: `/v3/users/52c797736d6b4d13ad5d661801f3b4ff` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:36.547977 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2f02f0-2840-4556-8a5b-567d485f04cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=52c797736d6b4d13ad5d661801f3b4ff)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:36.549913 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2f02f0-2840-4556-8a5b-567d485f04cd admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:36.612717 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5a2f02f0-2840-4556-8a5b-567d485f04cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52c797736d6b4d13ad5d661801f3b4ff'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:36.621316 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5a2f02f0-2840-4556-8a5b-567d485f04cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52c797736d6b4d13ad5d661801f3b4ff'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:36.622429 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5a2f02f0-2840-4556-8a5b-567d485f04cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52c797736d6b4d13ad5d661801f3b4ff'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:36.636327 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5a2f02f0-2840-4556-8a5b-567d485f04cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52c797736d6b4d13ad5d661801f3b4ff'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:36.637244 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5a2f02f0-2840-4556-8a5b-567d485f04cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52c797736d6b4d13ad5d661801f3b4ff'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:36.652933 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1143/2287] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:33:36 2022] DELETE /identity/v3/users/52c797736d6b4d13ad5d661801f3b4ff => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:33:36.947656 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fb5e41-5c14-43a7-8559-c8744b452fcf admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:36.949469 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-64fb5e41-5c14-43a7-8559-c8744b452fcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:36.966249 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fb5e41-5c14-43a7-8559-c8744b452fcf admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:37.046489 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c181f09-4fa9-4892-b37a-d08416a020b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:37.047469 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c181f09-4fa9-4892-b37a-d08416a020b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:37.047469 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c181f09-4fa9-4892-b37a-d08416a020b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:37.047745 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c181f09-4fa9-4892-b37a-d08416a020b1 admin admin] PATH_INFO: `/v3/projects/0c89c41c1ccb4ae89338efe6e5470489` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:37.049024 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c181f09-4fa9-4892-b37a-d08416a020b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c89c41c1ccb4ae89338efe6e5470489)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:37.051031 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c181f09-4fa9-4892-b37a-d08416a020b1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:37.094449 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:33:37.101664 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2c181f09-4fa9-4892-b37a-d08416a020b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c89c41c1ccb4ae89338efe6e5470489', 'request_id': 'req-2c181f09-4fa9-4892-b37a-d08416a020b1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:37.112703 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2c181f09-4fa9-4892-b37a-d08416a020b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c89c41c1ccb4ae89338efe6e5470489', 'request_id': 'req-2c181f09-4fa9-4892-b37a-d08416a020b1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:37.118519 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2c181f09-4fa9-4892-b37a-d08416a020b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c89c41c1ccb4ae89338efe6e5470489', 'request_id': 'req-2c181f09-4fa9-4892-b37a-d08416a020b1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:37.130269 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2c181f09-4fa9-4892-b37a-d08416a020b1 admin admin] The token cache is being invalidate because project 0c89c41c1ccb4ae89338efe6e5470489 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:33:37.130564 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2c181f09-4fa9-4892-b37a-d08416a020b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:37.131243 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2c181f09-4fa9-4892-b37a-d08416a020b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:37.132816 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1145/2288] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:33:36 2022] DELETE /identity/v3/projects/0c89c41c1ccb4ae89338efe6e5470489 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:33:37.138094 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94f548-a063-415f-a470-257ed5442aba None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:37.138661 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94f548-a063-415f-a470-257ed5442aba None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:37.138661 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94f548-a063-415f-a470-257ed5442aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:37.160408 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-6d94f548-a063-415f-a470-257ed5442aba None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:33:37.161718 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6d94f548-a063-415f-a470-257ed5442aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:37.210184 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:33:37.221820 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1144/2289] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:33:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:33:37.229764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c514f7c9-37f6-4993-a3d7-aa11cbb22f9d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:37.230032 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c514f7c9-37f6-4993-a3d7-aa11cbb22f9d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:37.230305 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c514f7c9-37f6-4993-a3d7-aa11cbb22f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:37.250332 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c514f7c9-37f6-4993-a3d7-aa11cbb22f9d None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:33:37.251588 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c514f7c9-37f6-4993-a3d7-aa11cbb22f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:37.261932 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1146/2290] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:33:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:33:37.271807 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94f548-a063-415f-a470-257ed5442aba None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:37.288546 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94f548-a063-415f-a470-257ed5442aba None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:37.297320 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a015306f-2462-4be3-82b7-ded5f90453f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:37.297905 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a015306f-2462-4be3-82b7-ded5f90453f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:37.298186 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a015306f-2462-4be3-82b7-ded5f90453f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:37.298455 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a015306f-2462-4be3-82b7-ded5f90453f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:37.299211 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a015306f-2462-4be3-82b7-ded5f90453f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:33:37.299515 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a015306f-2462-4be3-82b7-ded5f90453f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:37.301130 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a015306f-2462-4be3-82b7-ded5f90453f3 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:37.313391 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1145/2291] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:33:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:33:37.318404 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c514f7c9-37f6-4993-a3d7-aa11cbb22f9d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:37.331329 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c514f7c9-37f6-4993-a3d7-aa11cbb22f9d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:37.336108 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a26957-eb0f-4d79-a548-3d04500b9f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:37.336747 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a26957-eb0f-4d79-a548-3d04500b9f4a admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:37.337022 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a26957-eb0f-4d79-a548-3d04500b9f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:37.337290 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a26957-eb0f-4d79-a548-3d04500b9f4a admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:37.337980 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a26957-eb0f-4d79-a548-3d04500b9f4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:37.339930 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a26957-eb0f-4d79-a548-3d04500b9f4a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:37.371791 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1147/2292] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:33:37 2022] POST /identity/v3/projects => generated 387 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:33:37.376203 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a015306f-2462-4be3-82b7-ded5f90453f3 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:37.387614 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a015306f-2462-4be3-82b7-ded5f90453f3 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:37.452732 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eeae37-8b1b-4efa-ac4d-737943dfd909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:37.454723 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eeae37-8b1b-4efa-ac4d-737943dfd909 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:37.457166 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eeae37-8b1b-4efa-ac4d-737943dfd909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:37.457166 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eeae37-8b1b-4efa-ac4d-737943dfd909 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:37.458095 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eeae37-8b1b-4efa-ac4d-737943dfd909 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:37.459776 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eeae37-8b1b-4efa-ac4d-737943dfd909 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:37.484086 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1146/2293] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:33:37 2022] POST /identity/v3/users => generated 294 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:33:37.488722 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a26957-eb0f-4d79-a548-3d04500b9f4a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:37.499480 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a26957-eb0f-4d79-a548-3d04500b9f4a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:37.504425 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef884a3-c214-4ea0-985a-ba1452516497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:37.505040 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef884a3-c214-4ea0-985a-ba1452516497 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:37.505316 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef884a3-c214-4ea0-985a-ba1452516497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:37.505586 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef884a3-c214-4ea0-985a-ba1452516497 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:37.506273 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef884a3-c214-4ea0-985a-ba1452516497 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:37.507901 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef884a3-c214-4ea0-985a-ba1452516497 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:37.518302 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1148/2294] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:33:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 10:33:37.522445 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eeae37-8b1b-4efa-ac4d-737943dfd909 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:37.533251 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eeae37-8b1b-4efa-ac4d-737943dfd909 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:37.538929 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5403f8db-c1d4-4600-a709-0937f560132e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:37.539583 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403f8db-c1d4-4600-a709-0937f560132e admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:37.539863 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403f8db-c1d4-4600-a709-0937f560132e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:37.540134 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403f8db-c1d4-4600-a709-0937f560132e admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:37.541559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5403f8db-c1d4-4600-a709-0937f560132e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:37.543643 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5403f8db-c1d4-4600-a709-0937f560132e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:37.555602 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1147/2295] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:33:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:33:37.560373 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef884a3-c214-4ea0-985a-ba1452516497 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:37.571776 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef884a3-c214-4ea0-985a-ba1452516497 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:37.576709 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf5b92-a76e-4dbe-9f87-56ff0b52c580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:37.577386 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf5b92-a76e-4dbe-9f87-56ff0b52c580 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:37.577724 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf5b92-a76e-4dbe-9f87-56ff0b52c580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:37.578154 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf5b92-a76e-4dbe-9f87-56ff0b52c580 admin admin] PATH_INFO: `/v3/projects/4da74ea74e744c12a8f04e07585ace16/users/882d28e6082e434daab5ec011ff2451a/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:37.594422 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bf5b92-a76e-4dbe-9f87-56ff0b52c580 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4da74ea74e744c12a8f04e07585ace16, user_id=882d28e6082e434daab5ec011ff2451a, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:37.597103 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bf5b92-a76e-4dbe-9f87-56ff0b52c580 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:37.614380 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1149/2296] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:33:37 2022] PUT /identity/v3/projects/4da74ea74e744c12a8f04e07585ace16/users/882d28e6082e434daab5ec011ff2451a/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:33:37.624943 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7b33d-91ca-407b-a1e5-092018e02a98 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:37.625283 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7b33d-91ca-407b-a1e5-092018e02a98 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:37.625564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7b33d-91ca-407b-a1e5-092018e02a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:37.656952 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-3db7b33d-91ca-407b-a1e5-092018e02a98 None None] MFA Rules not processed for user `882d28e6082e434daab5ec011ff2451a`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:33:37.658165 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3db7b33d-91ca-407b-a1e5-092018e02a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:37.726385 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1148/2297] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:33:37 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:33:37.731379 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf5b92-a76e-4dbe-9f87-56ff0b52c580 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:37.802586 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf5b92-a76e-4dbe-9f87-56ff0b52c580 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:37.808391 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe8b2d0-c8c7-4b90-a61e-aebe074dc19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:37.809137 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8b2d0-c8c7-4b90-a61e-aebe074dc19c admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:37.809532 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8b2d0-c8c7-4b90-a61e-aebe074dc19c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:37.810354 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8b2d0-c8c7-4b90-a61e-aebe074dc19c admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:37.811166 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe8b2d0-c8c7-4b90-a61e-aebe074dc19c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:37.813475 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe8b2d0-c8c7-4b90-a61e-aebe074dc19c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:37.848401 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1150/2298] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:33:37 2022] POST /identity/v3/projects => generated 385 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:33:37.852224 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db7b33d-91ca-407b-a1e5-092018e02a98 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:37.865933 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db7b33d-91ca-407b-a1e5-092018e02a98 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:37.921703 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:33:37.933621 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb4ff0f-2240-438c-a29e-af8326d1cc32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:37.934322 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb4ff0f-2240-438c-a29e-af8326d1cc32 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:37.934691 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb4ff0f-2240-438c-a29e-af8326d1cc32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:37.935059 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb4ff0f-2240-438c-a29e-af8326d1cc32 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:37.935789 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb4ff0f-2240-438c-a29e-af8326d1cc32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:37.937989 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb4ff0f-2240-438c-a29e-af8326d1cc32 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:37.965654 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1149/2299] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:33:37 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:33:37.967086 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe8b2d0-c8c7-4b90-a61e-aebe074dc19c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:37.977594 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe8b2d0-c8c7-4b90-a61e-aebe074dc19c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:37.982073 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef961196-0b96-4462-b482-6f1a95468831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:37.982673 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef961196-0b96-4462-b482-6f1a95468831 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:37.983096 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef961196-0b96-4462-b482-6f1a95468831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:37.983369 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef961196-0b96-4462-b482-6f1a95468831 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:37.984105 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef961196-0b96-4462-b482-6f1a95468831 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:37.986376 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef961196-0b96-4462-b482-6f1a95468831 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:37.997096 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1151/2300] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:33:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:33:38.004660 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb4ff0f-2240-438c-a29e-af8326d1cc32 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:38.022144 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb4ff0f-2240-438c-a29e-af8326d1cc32 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:38.028291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285d2595-8be0-4c4d-9ec9-35dbfe0c889c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:38.028911 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d2595-8be0-4c4d-9ec9-35dbfe0c889c admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:38.029191 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d2595-8be0-4c4d-9ec9-35dbfe0c889c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:38.029477 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d2595-8be0-4c4d-9ec9-35dbfe0c889c admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:38.030177 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285d2595-8be0-4c4d-9ec9-35dbfe0c889c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:38.032117 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285d2595-8be0-4c4d-9ec9-35dbfe0c889c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:38.043208 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1150/2301] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:33:38 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:33:38.048353 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef961196-0b96-4462-b482-6f1a95468831 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:38.059503 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef961196-0b96-4462-b482-6f1a95468831 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:38.065155 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd91e3d-4b91-40ee-bffb-cf0543c5738a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:38.065829 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd91e3d-4b91-40ee-bffb-cf0543c5738a admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:38.066103 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd91e3d-4b91-40ee-bffb-cf0543c5738a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:38.066372 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd91e3d-4b91-40ee-bffb-cf0543c5738a admin admin] PATH_INFO: `/v3/projects/0d7fb7dc9b514a049eec4fe3eb6a0762/users/96af2d24acc0443d96a7ea48ea89652f/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:38.081787 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd91e3d-4b91-40ee-bffb-cf0543c5738a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d7fb7dc9b514a049eec4fe3eb6a0762, user_id=96af2d24acc0443d96a7ea48ea89652f, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:38.084496 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd91e3d-4b91-40ee-bffb-cf0543c5738a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:38.100716 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1152/2302] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:33:38 2022] PUT /identity/v3/projects/0d7fb7dc9b514a049eec4fe3eb6a0762/users/96af2d24acc0443d96a7ea48ea89652f/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:33:38.111266 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e9e26-e50e-43bb-896c-370d71831f6f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:38.111560 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e9e26-e50e-43bb-896c-370d71831f6f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:38.111833 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e9e26-e50e-43bb-896c-370d71831f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:38.143616 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-920e9e26-e50e-43bb-896c-370d71831f6f None None] MFA Rules not processed for user `96af2d24acc0443d96a7ea48ea89652f`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:33:38.144962 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-920e9e26-e50e-43bb-896c-370d71831f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:38.214983 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1151/2303] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:33:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:33:38.220247 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd91e3d-4b91-40ee-bffb-cf0543c5738a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:38.238360 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:33:38.289393 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd91e3d-4b91-40ee-bffb-cf0543c5738a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:38.296813 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676431c4-be2c-435f-bac0-5539c22905f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:38.297716 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676431c4-be2c-435f-bac0-5539c22905f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:38.297989 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676431c4-be2c-435f-bac0-5539c22905f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:38.298241 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676431c4-be2c-435f-bac0-5539c22905f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:38.298944 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676431c4-be2c-435f-bac0-5539c22905f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:38.300869 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676431c4-be2c-435f-bac0-5539c22905f8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:38.331181 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1153/2304] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:33:38 2022] POST /identity/v3/projects => generated 387 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:33:38.335224 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920e9e26-e50e-43bb-896c-370d71831f6f None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:38.347367 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920e9e26-e50e-43bb-896c-370d71831f6f None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:38.412284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9308481b-fb82-42ae-b94e-6391fa1c9b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:38.412924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308481b-fb82-42ae-b94e-6391fa1c9b69 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:38.413203 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308481b-fb82-42ae-b94e-6391fa1c9b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:38.413470 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308481b-fb82-42ae-b94e-6391fa1c9b69 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:38.414193 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9308481b-fb82-42ae-b94e-6391fa1c9b69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:38.416193 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9308481b-fb82-42ae-b94e-6391fa1c9b69 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:38.442183 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1152/2305] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:33:38 2022] POST /identity/v3/users => generated 300 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:33:38.446465 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676431c4-be2c-435f-bac0-5539c22905f8 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:38.457683 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676431c4-be2c-435f-bac0-5539c22905f8 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:38.463863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76c759-36fc-4e83-bb8e-1d2be0465b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:38.464501 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76c759-36fc-4e83-bb8e-1d2be0465b9e admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:38.464784 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76c759-36fc-4e83-bb8e-1d2be0465b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:38.465054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76c759-36fc-4e83-bb8e-1d2be0465b9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:38.465768 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76c759-36fc-4e83-bb8e-1d2be0465b9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:38.467449 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76c759-36fc-4e83-bb8e-1d2be0465b9e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:38.479344 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1154/2306] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:33:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:33:38.483958 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9308481b-fb82-42ae-b94e-6391fa1c9b69 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:38.495102 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9308481b-fb82-42ae-b94e-6391fa1c9b69 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:38.499947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced9e916-d7fb-49a3-8311-e81af7257627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:38.500631 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced9e916-d7fb-49a3-8311-e81af7257627 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:38.500909 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced9e916-d7fb-49a3-8311-e81af7257627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:38.501181 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced9e916-d7fb-49a3-8311-e81af7257627 admin admin] PATH_INFO: `/v3/projects/91fed83839ad447393b22c0db3b7e352/users/d888d587692740ffbcad8af210aea8ce/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:38.519769 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced9e916-d7fb-49a3-8311-e81af7257627 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91fed83839ad447393b22c0db3b7e352, user_id=d888d587692740ffbcad8af210aea8ce, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:38.522438 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced9e916-d7fb-49a3-8311-e81af7257627 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:38.540763 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1153/2307] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:33:38 2022] PUT /identity/v3/projects/91fed83839ad447393b22c0db3b7e352/users/d888d587692740ffbcad8af210aea8ce/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:33:38.545184 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76c759-36fc-4e83-bb8e-1d2be0465b9e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:38.556210 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76c759-36fc-4e83-bb8e-1d2be0465b9e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:38.617658 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d7bacb-b976-4b66-be1f-be64187bf3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:38.618708 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7bacb-b976-4b66-be1f-be64187bf3cd admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:38.619294 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7bacb-b976-4b66-be1f-be64187bf3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:38.619594 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7bacb-b976-4b66-be1f-be64187bf3cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:38.620320 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d7bacb-b976-4b66-be1f-be64187bf3cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:38.621952 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d7bacb-b976-4b66-be1f-be64187bf3cd admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:38.636252 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1155/2308] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:33:38 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 10:33:38.637439 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced9e916-d7fb-49a3-8311-e81af7257627 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:38.648547 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced9e916-d7fb-49a3-8311-e81af7257627 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:38.653177 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45307d1d-8d94-4e15-9b63-83f8b34c1c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:38.653851 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45307d1d-8d94-4e15-9b63-83f8b34c1c5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:38.654134 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45307d1d-8d94-4e15-9b63-83f8b34c1c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:38.654408 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45307d1d-8d94-4e15-9b63-83f8b34c1c5d admin admin] PATH_INFO: `/v3/projects/91fed83839ad447393b22c0db3b7e352/users/d888d587692740ffbcad8af210aea8ce/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:38.656400 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45307d1d-8d94-4e15-9b63-83f8b34c1c5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=91fed83839ad447393b22c0db3b7e352, user_id=d888d587692740ffbcad8af210aea8ce, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:38.659329 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45307d1d-8d94-4e15-9b63-83f8b34c1c5d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:38.669981 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1154/2309] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:33:38 2022] PUT /identity/v3/projects/91fed83839ad447393b22c0db3b7e352/users/d888d587692740ffbcad8af210aea8ce/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:33:38.681315 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5962e592-b4ad-4cee-a15c-57a3744e3653 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:38.681607 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5962e592-b4ad-4cee-a15c-57a3744e3653 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:38.681891 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5962e592-b4ad-4cee-a15c-57a3744e3653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:38.712680 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-5962e592-b4ad-4cee-a15c-57a3744e3653 None None] MFA Rules not processed for user `d888d587692740ffbcad8af210aea8ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:33:38.713961 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5962e592-b4ad-4cee-a15c-57a3744e3653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:38.784263 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1156/2310] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:33:38 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:33:38.795148 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45307d1d-8d94-4e15-9b63-83f8b34c1c5d admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:38.797002 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-45307d1d-8d94-4e15-9b63-83f8b34c1c5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:38.812657 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:33:38.857226 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45307d1d-8d94-4e15-9b63-83f8b34c1c5d admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:38.861458 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca5686-6517-445b-9890-a8e08014d434 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:38.862090 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca5686-6517-445b-9890-a8e08014d434 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:38.862400 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca5686-6517-445b-9890-a8e08014d434 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:38.862670 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca5686-6517-445b-9890-a8e08014d434 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:38.872108 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca5686-6517-445b-9890-a8e08014d434 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:38.873540 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca5686-6517-445b-9890-a8e08014d434 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:38.884111 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1155/2311] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:33:38 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:33:39.504346 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5962e592-b4ad-4cee-a15c-57a3744e3653 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:39.506377 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5962e592-b4ad-4cee-a15c-57a3744e3653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:39.519176 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5962e592-b4ad-4cee-a15c-57a3744e3653 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:39.527540 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e24e367-6e71-4f32-a010-a39249d43aa6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:39.531843 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24e367-6e71-4f32-a010-a39249d43aa6 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:39.532097 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24e367-6e71-4f32-a010-a39249d43aa6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:39.532376 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24e367-6e71-4f32-a010-a39249d43aa6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:39.544169 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:33:39.606936 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e24e367-6e71-4f32-a010-a39249d43aa6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:39.607394 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e24e367-6e71-4f32-a010-a39249d43aa6 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:39.627090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1157/2312] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:33:39 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:33:43.729871 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca5686-6517-445b-9890-a8e08014d434 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:43.739190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca5686-6517-445b-9890-a8e08014d434 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:43.743292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c42565-4060-4896-a51c-0a6cb93610db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:43.743824 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c42565-4060-4896-a51c-0a6cb93610db service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:43.744095 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c42565-4060-4896-a51c-0a6cb93610db service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:43.744330 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c42565-4060-4896-a51c-0a6cb93610db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:43.800136 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c42565-4060-4896-a51c-0a6cb93610db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:43.801557 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c42565-4060-4896-a51c-0a6cb93610db service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:43.810451 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1156/2313] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:33:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:33:46.003480 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e24e367-6e71-4f32-a010-a39249d43aa6 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:46.073450 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e24e367-6e71-4f32-a010-a39249d43aa6 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:46.081450 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c08239-05a5-4daa-a2bd-095354e6a999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:46.082190 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c08239-05a5-4daa-a2bd-095354e6a999 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:46.082455 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c08239-05a5-4daa-a2bd-095354e6a999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:46.082821 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c08239-05a5-4daa-a2bd-095354e6a999 admin admin] PATH_INFO: `/v3/users/882d28e6082e434daab5ec011ff2451a` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:46.084098 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c08239-05a5-4daa-a2bd-095354e6a999 admin admin] RBAC: Authorizing `identity:delete_user(user_id=882d28e6082e434daab5ec011ff2451a)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:46.086021 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c08239-05a5-4daa-a2bd-095354e6a999 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:46.143217 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-33c08239-05a5-4daa-a2bd-095354e6a999 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '882d28e6082e434daab5ec011ff2451a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:46.143928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-33c08239-05a5-4daa-a2bd-095354e6a999 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '882d28e6082e434daab5ec011ff2451a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:46.144555 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-33c08239-05a5-4daa-a2bd-095354e6a999 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '882d28e6082e434daab5ec011ff2451a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:46.153800 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-33c08239-05a5-4daa-a2bd-095354e6a999 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '882d28e6082e434daab5ec011ff2451a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:46.164735 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-33c08239-05a5-4daa-a2bd-095354e6a999 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '882d28e6082e434daab5ec011ff2451a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:46.188037 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1158/2314] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:33:46 2022] DELETE /identity/v3/users/882d28e6082e434daab5ec011ff2451a => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:33:46.192983 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc2c62-c4c8-4538-972a-aaa866dbeb62 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:46.193475 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc2c62-c4c8-4538-972a-aaa866dbeb62 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:46.193587 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc2c62-c4c8-4538-972a-aaa866dbeb62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:46.214067 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b1dc2c62-c4c8-4538-972a-aaa866dbeb62 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:33:46.215307 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b1dc2c62-c4c8-4538-972a-aaa866dbeb62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:46.287063 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1157/2315] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:33:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:33:46.297379 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c08239-05a5-4daa-a2bd-095354e6a999 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:46.298930 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-33c08239-05a5-4daa-a2bd-095354e6a999 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:46.332495 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:33:46.360750 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c08239-05a5-4daa-a2bd-095354e6a999 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:46.365903 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689db140-b78d-42af-a526-783a37202367 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:46.366531 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689db140-b78d-42af-a526-783a37202367 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:46.366812 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689db140-b78d-42af-a526-783a37202367 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:46.367115 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689db140-b78d-42af-a526-783a37202367 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:46.377114 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689db140-b78d-42af-a526-783a37202367 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:46.378570 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689db140-b78d-42af-a526-783a37202367 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:46.388876 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1159/2316] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:33:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:33:46.675173 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dc2c62-c4c8-4538-972a-aaa866dbeb62 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:46.676986 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b1dc2c62-c4c8-4538-972a-aaa866dbeb62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:46.689637 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dc2c62-c4c8-4538-972a-aaa866dbeb62 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:46.695421 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d988a9eb-01d8-46f2-894c-2c576f932821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:46.696096 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d988a9eb-01d8-46f2-894c-2c576f932821 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:46.696372 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d988a9eb-01d8-46f2-894c-2c576f932821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:46.696640 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d988a9eb-01d8-46f2-894c-2c576f932821 admin admin] PATH_INFO: `/v3/projects/4da74ea74e744c12a8f04e07585ace16` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:46.697849 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d988a9eb-01d8-46f2-894c-2c576f932821 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4da74ea74e744c12a8f04e07585ace16)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:46.699814 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d988a9eb-01d8-46f2-894c-2c576f932821 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:46.757072 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d988a9eb-01d8-46f2-894c-2c576f932821 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4da74ea74e744c12a8f04e07585ace16', 'request_id': 'req-d988a9eb-01d8-46f2-894c-2c576f932821'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:46.758318 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:33:46.764206 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d988a9eb-01d8-46f2-894c-2c576f932821 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4da74ea74e744c12a8f04e07585ace16', 'request_id': 'req-d988a9eb-01d8-46f2-894c-2c576f932821'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:46.774317 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d988a9eb-01d8-46f2-894c-2c576f932821 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4da74ea74e744c12a8f04e07585ace16', 'request_id': 'req-d988a9eb-01d8-46f2-894c-2c576f932821'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:46.788546 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d988a9eb-01d8-46f2-894c-2c576f932821 admin admin] The token cache is being invalidate because project 4da74ea74e744c12a8f04e07585ace16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:33:46.788825 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d988a9eb-01d8-46f2-894c-2c576f932821 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:46.789503 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d988a9eb-01d8-46f2-894c-2c576f932821 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:46.791141 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1158/2317] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:33:46 2022] DELETE /identity/v3/projects/4da74ea74e744c12a8f04e07585ace16 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:33:46.795355 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689db140-b78d-42af-a526-783a37202367 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:46.806985 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689db140-b78d-42af-a526-783a37202367 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:46.877822 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a420c-8945-4550-9c18-c51fd8cfd92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:46.878762 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a420c-8945-4550-9c18-c51fd8cfd92f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:46.879149 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a420c-8945-4550-9c18-c51fd8cfd92f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:46.879629 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a420c-8945-4550-9c18-c51fd8cfd92f admin admin] PATH_INFO: `/v3/users/96af2d24acc0443d96a7ea48ea89652f` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:46.880901 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a420c-8945-4550-9c18-c51fd8cfd92f admin admin] RBAC: Authorizing `identity:delete_user(user_id=96af2d24acc0443d96a7ea48ea89652f)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:46.882857 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a420c-8945-4550-9c18-c51fd8cfd92f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:46.934006 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-195a420c-8945-4550-9c18-c51fd8cfd92f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96af2d24acc0443d96a7ea48ea89652f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:46.934706 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-195a420c-8945-4550-9c18-c51fd8cfd92f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96af2d24acc0443d96a7ea48ea89652f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:46.935395 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-195a420c-8945-4550-9c18-c51fd8cfd92f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96af2d24acc0443d96a7ea48ea89652f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:46.946102 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-195a420c-8945-4550-9c18-c51fd8cfd92f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96af2d24acc0443d96a7ea48ea89652f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:46.956572 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-195a420c-8945-4550-9c18-c51fd8cfd92f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96af2d24acc0443d96a7ea48ea89652f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:46.975650 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1160/2318] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:33:46 2022] DELETE /identity/v3/users/96af2d24acc0443d96a7ea48ea89652f => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:33:47.265837 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d988a9eb-01d8-46f2-894c-2c576f932821 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:47.267351 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d988a9eb-01d8-46f2-894c-2c576f932821 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:47.282824 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d988a9eb-01d8-46f2-894c-2c576f932821 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:47.343727 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c03b7-c2da-481d-b9c0-e0bc5e0ef311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:47.344288 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c03b7-c2da-481d-b9c0-e0bc5e0ef311 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:47.344571 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c03b7-c2da-481d-b9c0-e0bc5e0ef311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:47.344825 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c03b7-c2da-481d-b9c0-e0bc5e0ef311 admin admin] PATH_INFO: `/v3/projects/0d7fb7dc9b514a049eec4fe3eb6a0762` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:47.345979 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98c03b7-c2da-481d-b9c0-e0bc5e0ef311 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d7fb7dc9b514a049eec4fe3eb6a0762)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:47.347996 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98c03b7-c2da-481d-b9c0-e0bc5e0ef311 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:47.398262 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f98c03b7-c2da-481d-b9c0-e0bc5e0ef311 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d7fb7dc9b514a049eec4fe3eb6a0762', 'request_id': 'req-f98c03b7-c2da-481d-b9c0-e0bc5e0ef311'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:47.404625 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f98c03b7-c2da-481d-b9c0-e0bc5e0ef311 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d7fb7dc9b514a049eec4fe3eb6a0762', 'request_id': 'req-f98c03b7-c2da-481d-b9c0-e0bc5e0ef311'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:47.413550 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f98c03b7-c2da-481d-b9c0-e0bc5e0ef311 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d7fb7dc9b514a049eec4fe3eb6a0762', 'request_id': 'req-f98c03b7-c2da-481d-b9c0-e0bc5e0ef311'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:47.424629 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f98c03b7-c2da-481d-b9c0-e0bc5e0ef311 admin admin] The token cache is being invalidate because project 0d7fb7dc9b514a049eec4fe3eb6a0762 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:33:47.424917 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f98c03b7-c2da-481d-b9c0-e0bc5e0ef311 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:47.425583 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f98c03b7-c2da-481d-b9c0-e0bc5e0ef311 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:47.427246 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1159/2319] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:33:47 2022] DELETE /identity/v3/projects/0d7fb7dc9b514a049eec4fe3eb6a0762 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:33:47.431409 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a420c-8945-4550-9c18-c51fd8cfd92f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:47.432850 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-195a420c-8945-4550-9c18-c51fd8cfd92f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:47.445483 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a420c-8945-4550-9c18-c51fd8cfd92f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:47.497872 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:33:47.510143 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d068b6-bb96-4475-b9e6-acd01cb4e77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:47.510795 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d068b6-bb96-4475-b9e6-acd01cb4e77e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:47.511122 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d068b6-bb96-4475-b9e6-acd01cb4e77e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:47.511399 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d068b6-bb96-4475-b9e6-acd01cb4e77e admin admin] PATH_INFO: `/v3/users/d888d587692740ffbcad8af210aea8ce` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:47.512584 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d068b6-bb96-4475-b9e6-acd01cb4e77e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d888d587692740ffbcad8af210aea8ce)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:47.514623 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d068b6-bb96-4475-b9e6-acd01cb4e77e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:47.565543 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-15d068b6-bb96-4475-b9e6-acd01cb4e77e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd888d587692740ffbcad8af210aea8ce'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:47.566164 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-15d068b6-bb96-4475-b9e6-acd01cb4e77e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd888d587692740ffbcad8af210aea8ce'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:47.566949 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-15d068b6-bb96-4475-b9e6-acd01cb4e77e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd888d587692740ffbcad8af210aea8ce'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:47.577882 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-15d068b6-bb96-4475-b9e6-acd01cb4e77e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd888d587692740ffbcad8af210aea8ce'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:47.585344 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-15d068b6-bb96-4475-b9e6-acd01cb4e77e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd888d587692740ffbcad8af210aea8ce'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:47.604312 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1161/2320] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:33:47 2022] DELETE /identity/v3/users/d888d587692740ffbcad8af210aea8ce => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:33:47.896730 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c03b7-c2da-481d-b9c0-e0bc5e0ef311 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:47.898684 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f98c03b7-c2da-481d-b9c0-e0bc5e0ef311 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:47.912020 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c03b7-c2da-481d-b9c0-e0bc5e0ef311 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:47.972607 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe0c8ef-c00a-4844-b186-d5009aa53a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:47.973315 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe0c8ef-c00a-4844-b186-d5009aa53a19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:47.973629 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe0c8ef-c00a-4844-b186-d5009aa53a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:47.973965 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe0c8ef-c00a-4844-b186-d5009aa53a19 admin admin] PATH_INFO: `/v3/projects/91fed83839ad447393b22c0db3b7e352` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:47.975193 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe0c8ef-c00a-4844-b186-d5009aa53a19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91fed83839ad447393b22c0db3b7e352)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:47.977116 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe0c8ef-c00a-4844-b186-d5009aa53a19 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:47.987665 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:33:48.026961 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7fe0c8ef-c00a-4844-b186-d5009aa53a19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91fed83839ad447393b22c0db3b7e352', 'request_id': 'req-7fe0c8ef-c00a-4844-b186-d5009aa53a19'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:48.032446 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7fe0c8ef-c00a-4844-b186-d5009aa53a19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91fed83839ad447393b22c0db3b7e352', 'request_id': 'req-7fe0c8ef-c00a-4844-b186-d5009aa53a19'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:48.041709 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7fe0c8ef-c00a-4844-b186-d5009aa53a19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91fed83839ad447393b22c0db3b7e352', 'request_id': 'req-7fe0c8ef-c00a-4844-b186-d5009aa53a19'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:48.054112 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7fe0c8ef-c00a-4844-b186-d5009aa53a19 admin admin] The token cache is being invalidate because project 91fed83839ad447393b22c0db3b7e352 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:33:48.054402 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7fe0c8ef-c00a-4844-b186-d5009aa53a19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:48.055073 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7fe0c8ef-c00a-4844-b186-d5009aa53a19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:48.056648 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1160/2321] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:33:47 2022] DELETE /identity/v3/projects/91fed83839ad447393b22c0db3b7e352 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:33:48.062829 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea7207-8439-4f6f-adc0-e538d9b86897 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:48.063147 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea7207-8439-4f6f-adc0-e538d9b86897 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:48.063421 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea7207-8439-4f6f-adc0-e538d9b86897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:48.083243 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-efea7207-8439-4f6f-adc0-e538d9b86897 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:33:48.084429 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-efea7207-8439-4f6f-adc0-e538d9b86897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:48.146711 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1162/2322] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:33:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:33:48.155910 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a6eff-6e7a-4233-9441-65db3a3b4b65 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:48.156199 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a6eff-6e7a-4233-9441-65db3a3b4b65 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:48.156475 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a6eff-6e7a-4233-9441-65db3a3b4b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:48.178323 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b54a6eff-6e7a-4233-9441-65db3a3b4b65 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:33:48.179550 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b54a6eff-6e7a-4233-9441-65db3a3b4b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:48.188382 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1161/2323] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:33:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:33:48.193091 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea7207-8439-4f6f-adc0-e538d9b86897 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:48.203476 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea7207-8439-4f6f-adc0-e538d9b86897 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:48.209133 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f7d75-0095-44fa-8d34-158f5dbc04e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:48.209752 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f7d75-0095-44fa-8d34-158f5dbc04e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:48.210038 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f7d75-0095-44fa-8d34-158f5dbc04e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:48.210337 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f7d75-0095-44fa-8d34-158f5dbc04e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:48.211070 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337f7d75-0095-44fa-8d34-158f5dbc04e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:33:48.211379 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337f7d75-0095-44fa-8d34-158f5dbc04e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:48.212991 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337f7d75-0095-44fa-8d34-158f5dbc04e4 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:48.224739 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1163/2324] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:33:48.229142 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54a6eff-6e7a-4233-9441-65db3a3b4b65 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:48.238283 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54a6eff-6e7a-4233-9441-65db3a3b4b65 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:48.243778 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db45fd52-1f80-4a82-948b-6fae6d23b805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:48.244388 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45fd52-1f80-4a82-948b-6fae6d23b805 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:48.244663 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45fd52-1f80-4a82-948b-6fae6d23b805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:48.244931 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45fd52-1f80-4a82-948b-6fae6d23b805 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:48.245592 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db45fd52-1f80-4a82-948b-6fae6d23b805 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:48.247552 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db45fd52-1f80-4a82-948b-6fae6d23b805 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:48.279106 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1162/2325] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:33:48 2022] POST /identity/v3/projects => generated 389 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:33:48.283369 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f7d75-0095-44fa-8d34-158f5dbc04e4 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:48.294259 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f7d75-0095-44fa-8d34-158f5dbc04e4 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:48.346852 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:33:48.354092 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9328f934-c96b-4e01-ad1d-680810c6ab03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:48.354796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328f934-c96b-4e01-ad1d-680810c6ab03 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:48.355122 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328f934-c96b-4e01-ad1d-680810c6ab03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:48.355659 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328f934-c96b-4e01-ad1d-680810c6ab03 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:48.356610 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9328f934-c96b-4e01-ad1d-680810c6ab03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:48.359321 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9328f934-c96b-4e01-ad1d-680810c6ab03 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:48.383926 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1164/2326] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:33:48 2022] POST /identity/v3/users => generated 295 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:33:48.388298 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db45fd52-1f80-4a82-948b-6fae6d23b805 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:48.399561 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db45fd52-1f80-4a82-948b-6fae6d23b805 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:48.405284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd4be5-c9e9-4f40-a281-e15e77e0d039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:48.405900 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd4be5-c9e9-4f40-a281-e15e77e0d039 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:48.406179 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd4be5-c9e9-4f40-a281-e15e77e0d039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:48.406449 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd4be5-c9e9-4f40-a281-e15e77e0d039 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:48.407187 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bd4be5-c9e9-4f40-a281-e15e77e0d039 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:48.408829 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bd4be5-c9e9-4f40-a281-e15e77e0d039 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:48.419133 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1163/2327] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:33:48.423317 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9328f934-c96b-4e01-ad1d-680810c6ab03 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:48.433761 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9328f934-c96b-4e01-ad1d-680810c6ab03 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:48.439198 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b24bf43-8a0a-4d85-b452-78010163d6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:48.439836 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b24bf43-8a0a-4d85-b452-78010163d6db admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:48.440112 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b24bf43-8a0a-4d85-b452-78010163d6db admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:48.440385 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b24bf43-8a0a-4d85-b452-78010163d6db admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:48.441076 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b24bf43-8a0a-4d85-b452-78010163d6db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:48.442771 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b24bf43-8a0a-4d85-b452-78010163d6db admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:48.454726 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1165/2328] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:33:48.459483 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd4be5-c9e9-4f40-a281-e15e77e0d039 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:48.470232 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd4be5-c9e9-4f40-a281-e15e77e0d039 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:48.475433 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b174ed6d-9744-451d-8c41-2454f0d913a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:48.476089 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174ed6d-9744-451d-8c41-2454f0d913a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:48.476369 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174ed6d-9744-451d-8c41-2454f0d913a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:48.476637 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174ed6d-9744-451d-8c41-2454f0d913a8 admin admin] PATH_INFO: `/v3/projects/d2ba7306b0044108a56f70dcfac58f8d/users/cf620f45bd96441fa3d381d5fe7a05f4/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:48.491586 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b174ed6d-9744-451d-8c41-2454f0d913a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2ba7306b0044108a56f70dcfac58f8d, user_id=cf620f45bd96441fa3d381d5fe7a05f4, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:48.494228 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b174ed6d-9744-451d-8c41-2454f0d913a8 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:48.508746 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1164/2329] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:33:48 2022] PUT /identity/v3/projects/d2ba7306b0044108a56f70dcfac58f8d/users/cf620f45bd96441fa3d381d5fe7a05f4/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:33:48.521332 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf54d6c-0dcd-444f-9538-db857c98362f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:48.521620 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf54d6c-0dcd-444f-9538-db857c98362f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:48.521892 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf54d6c-0dcd-444f-9538-db857c98362f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:48.551581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-0bf54d6c-0dcd-444f-9538-db857c98362f None None] MFA Rules not processed for user `cf620f45bd96441fa3d381d5fe7a05f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:33:48.553022 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0bf54d6c-0dcd-444f-9538-db857c98362f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:48.618140 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1166/2330] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:33:48 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:33:48.622847 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b174ed6d-9744-451d-8c41-2454f0d913a8 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:48.687128 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b174ed6d-9744-451d-8c41-2454f0d913a8 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:48.692793 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c28d2a7-e451-4232-bafd-4f07c9896735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:48.693463 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28d2a7-e451-4232-bafd-4f07c9896735 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:48.693815 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28d2a7-e451-4232-bafd-4f07c9896735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:48.694139 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28d2a7-e451-4232-bafd-4f07c9896735 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:48.694850 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c28d2a7-e451-4232-bafd-4f07c9896735 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:48.696884 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c28d2a7-e451-4232-bafd-4f07c9896735 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:48.730606 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1165/2331] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:33:48 2022] POST /identity/v3/projects => generated 389 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:33:48.735704 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf54d6c-0dcd-444f-9538-db857c98362f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:48.748788 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf54d6c-0dcd-444f-9538-db857c98362f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:48.812631 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54eb0f0-8cad-487d-810b-bac5e315e314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:48.813255 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54eb0f0-8cad-487d-810b-bac5e315e314 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:48.813537 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54eb0f0-8cad-487d-810b-bac5e315e314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:48.813811 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54eb0f0-8cad-487d-810b-bac5e315e314 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:48.814477 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54eb0f0-8cad-487d-810b-bac5e315e314 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:48.816540 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54eb0f0-8cad-487d-810b-bac5e315e314 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:48.842715 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1167/2332] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:33:48 2022] POST /identity/v3/users => generated 301 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:33:48.846059 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c28d2a7-e451-4232-bafd-4f07c9896735 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:48.857143 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c28d2a7-e451-4232-bafd-4f07c9896735 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:48.861868 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee322b7-5b1e-4f22-adf2-cde18a9d732d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:48.862483 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee322b7-5b1e-4f22-adf2-cde18a9d732d admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:48.862744 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee322b7-5b1e-4f22-adf2-cde18a9d732d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:48.863046 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee322b7-5b1e-4f22-adf2-cde18a9d732d admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:48.864051 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee322b7-5b1e-4f22-adf2-cde18a9d732d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:48.866483 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee322b7-5b1e-4f22-adf2-cde18a9d732d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:48.883196 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1166/2333] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:33:48.889489 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54eb0f0-8cad-487d-810b-bac5e315e314 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:48.902193 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54eb0f0-8cad-487d-810b-bac5e315e314 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:48.907842 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20091fd-475b-4cfe-9fa2-537e82123ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:48.908508 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20091fd-475b-4cfe-9fa2-537e82123ad4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:48.908804 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20091fd-475b-4cfe-9fa2-537e82123ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:48.909063 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20091fd-475b-4cfe-9fa2-537e82123ad4 admin admin] PATH_INFO: `/v3/projects/0c0bbd6f6c9f47048911bfe3c1ba19f2/users/548f411fb3a24eaea0043a7b3438584d/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:48.924886 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20091fd-475b-4cfe-9fa2-537e82123ad4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c0bbd6f6c9f47048911bfe3c1ba19f2, user_id=548f411fb3a24eaea0043a7b3438584d, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:48.927733 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20091fd-475b-4cfe-9fa2-537e82123ad4 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:48.943940 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1168/2334] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:33:48 2022] PUT /identity/v3/projects/0c0bbd6f6c9f47048911bfe3c1ba19f2/users/548f411fb3a24eaea0043a7b3438584d/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:33:48.948070 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee322b7-5b1e-4f22-adf2-cde18a9d732d admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:48.958632 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee322b7-5b1e-4f22-adf2-cde18a9d732d admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:48.964562 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:33:49.023075 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07daa73-5d51-4608-a10d-35b8b07a088e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:49.023698 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07daa73-5d51-4608-a10d-35b8b07a088e admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:49.023978 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07daa73-5d51-4608-a10d-35b8b07a088e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:49.024253 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07daa73-5d51-4608-a10d-35b8b07a088e admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:49.024945 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07daa73-5d51-4608-a10d-35b8b07a088e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:49.026537 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07daa73-5d51-4608-a10d-35b8b07a088e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:49.036422 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1167/2335] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 10:33:49.041097 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20091fd-475b-4cfe-9fa2-537e82123ad4 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:49.052046 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20091fd-475b-4cfe-9fa2-537e82123ad4 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:49.057488 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64803a55-15be-49ce-918a-afa6ce607bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:49.058163 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64803a55-15be-49ce-918a-afa6ce607bda admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:49.058448 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64803a55-15be-49ce-918a-afa6ce607bda admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:49.058726 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64803a55-15be-49ce-918a-afa6ce607bda admin admin] PATH_INFO: `/v3/projects/0c0bbd6f6c9f47048911bfe3c1ba19f2/users/548f411fb3a24eaea0043a7b3438584d/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:49.061039 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64803a55-15be-49ce-918a-afa6ce607bda admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c0bbd6f6c9f47048911bfe3c1ba19f2, user_id=548f411fb3a24eaea0043a7b3438584d, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:49.063756 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64803a55-15be-49ce-918a-afa6ce607bda admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:49.074783 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1169/2336] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:33:49 2022] PUT /identity/v3/projects/0c0bbd6f6c9f47048911bfe3c1ba19f2/users/548f411fb3a24eaea0043a7b3438584d/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:33:49.085351 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a69c1c-5566-4242-b86a-1a930efaf522 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:49.085639 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a69c1c-5566-4242-b86a-1a930efaf522 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:49.085915 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a69c1c-5566-4242-b86a-1a930efaf522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:49.115063 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-98a69c1c-5566-4242-b86a-1a930efaf522 None None] MFA Rules not processed for user `548f411fb3a24eaea0043a7b3438584d`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:33:49.116271 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-98a69c1c-5566-4242-b86a-1a930efaf522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:49.189778 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1168/2337] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:33:49 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:33:49.201695 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64803a55-15be-49ce-918a-afa6ce607bda admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:49.203891 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-64803a55-15be-49ce-918a-afa6ce607bda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:49.227294 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:33:49.266233 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64803a55-15be-49ce-918a-afa6ce607bda admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:49.270416 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d003dc0-2c15-4aa8-9991-d914e4417fed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:49.271087 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d003dc0-2c15-4aa8-9991-d914e4417fed service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:49.271370 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d003dc0-2c15-4aa8-9991-d914e4417fed service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:49.271645 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d003dc0-2c15-4aa8-9991-d914e4417fed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:49.281402 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d003dc0-2c15-4aa8-9991-d914e4417fed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:49.282845 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d003dc0-2c15-4aa8-9991-d914e4417fed service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:49.292463 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1170/2338] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:33:49 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:33:49.431754 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a69c1c-5566-4242-b86a-1a930efaf522 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:49.494893 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a69c1c-5566-4242-b86a-1a930efaf522 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:49.500133 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a4123-114a-4b06-bd9a-504dde734094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:49.500779 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a4123-114a-4b06-bd9a-504dde734094 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:49.501055 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a4123-114a-4b06-bd9a-504dde734094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:49.501343 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a4123-114a-4b06-bd9a-504dde734094 admin admin] PATH_INFO: `/v3/users/cf620f45bd96441fa3d381d5fe7a05f4` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:49.502561 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296a4123-114a-4b06-bd9a-504dde734094 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf620f45bd96441fa3d381d5fe7a05f4)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:49.504875 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296a4123-114a-4b06-bd9a-504dde734094 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:49.546220 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:33:49.560300 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-296a4123-114a-4b06-bd9a-504dde734094 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf620f45bd96441fa3d381d5fe7a05f4'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:49.568827 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-296a4123-114a-4b06-bd9a-504dde734094 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf620f45bd96441fa3d381d5fe7a05f4'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:49.569440 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-296a4123-114a-4b06-bd9a-504dde734094 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf620f45bd96441fa3d381d5fe7a05f4'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:49.581441 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-296a4123-114a-4b06-bd9a-504dde734094 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf620f45bd96441fa3d381d5fe7a05f4'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:49.582153 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-296a4123-114a-4b06-bd9a-504dde734094 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf620f45bd96441fa3d381d5fe7a05f4'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:49.595536 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1169/2339] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:33:49 2022] DELETE /identity/v3/users/cf620f45bd96441fa3d381d5fe7a05f4 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:33:49.600450 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8719a-7b56-4883-9885-5de7fd9cb5bf None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:49.600920 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8719a-7b56-4883-9885-5de7fd9cb5bf None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:49.601014 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8719a-7b56-4883-9885-5de7fd9cb5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:49.620873 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c8c8719a-7b56-4883-9885-5de7fd9cb5bf None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:33:49.622072 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c8c8719a-7b56-4883-9885-5de7fd9cb5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:49.683789 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1171/2340] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:33:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:33:49.696301 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a4123-114a-4b06-bd9a-504dde734094 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:49.698804 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-296a4123-114a-4b06-bd9a-504dde734094 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:49.765384 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a4123-114a-4b06-bd9a-504dde734094 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:49.769304 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf26e7-f359-484f-88d9-e419da745513 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:49.769924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf26e7-f359-484f-88d9-e419da745513 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:49.770190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf26e7-f359-484f-88d9-e419da745513 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:49.770486 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf26e7-f359-484f-88d9-e419da745513 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:49.780451 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaf26e7-f359-484f-88d9-e419da745513 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:49.781879 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaf26e7-f359-484f-88d9-e419da745513 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:49.791786 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1170/2341] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:33:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:33:50.089674 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c8719a-7b56-4883-9885-5de7fd9cb5bf None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:50.091205 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c8c8719a-7b56-4883-9885-5de7fd9cb5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:50.103566 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c8719a-7b56-4883-9885-5de7fd9cb5bf None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:50.109208 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25a6dcd-d806-486e-8373-38ef646458f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:50.109874 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a6dcd-d806-486e-8373-38ef646458f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:50.110147 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a6dcd-d806-486e-8373-38ef646458f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:50.110412 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a6dcd-d806-486e-8373-38ef646458f8 admin admin] PATH_INFO: `/v3/projects/d2ba7306b0044108a56f70dcfac58f8d` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:50.112498 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25a6dcd-d806-486e-8373-38ef646458f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2ba7306b0044108a56f70dcfac58f8d)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:50.114583 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25a6dcd-d806-486e-8373-38ef646458f8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:50.171871 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a25a6dcd-d806-486e-8373-38ef646458f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2ba7306b0044108a56f70dcfac58f8d', 'request_id': 'req-a25a6dcd-d806-486e-8373-38ef646458f8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:50.181976 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a25a6dcd-d806-486e-8373-38ef646458f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2ba7306b0044108a56f70dcfac58f8d', 'request_id': 'req-a25a6dcd-d806-486e-8373-38ef646458f8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:50.189445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a25a6dcd-d806-486e-8373-38ef646458f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2ba7306b0044108a56f70dcfac58f8d', 'request_id': 'req-a25a6dcd-d806-486e-8373-38ef646458f8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:50.201415 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a25a6dcd-d806-486e-8373-38ef646458f8 admin admin] The token cache is being invalidate because project d2ba7306b0044108a56f70dcfac58f8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:33:50.201710 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a25a6dcd-d806-486e-8373-38ef646458f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:50.202443 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a25a6dcd-d806-486e-8373-38ef646458f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:50.204406 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1172/2342] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:33:50 2022] DELETE /identity/v3/projects/d2ba7306b0044108a56f70dcfac58f8d => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:33:50.208272 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf26e7-f359-484f-88d9-e419da745513 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:50.219319 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf26e7-f359-484f-88d9-e419da745513 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:50.291029 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0fc452-b771-463d-bc4a-7ca56563b6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:50.291625 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0fc452-b771-463d-bc4a-7ca56563b6b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:50.291902 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0fc452-b771-463d-bc4a-7ca56563b6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:50.292169 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0fc452-b771-463d-bc4a-7ca56563b6b8 admin admin] PATH_INFO: `/v3/users/548f411fb3a24eaea0043a7b3438584d` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:50.293320 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0fc452-b771-463d-bc4a-7ca56563b6b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=548f411fb3a24eaea0043a7b3438584d)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:50.295290 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0fc452-b771-463d-bc4a-7ca56563b6b8 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:50.348210 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-da0fc452-b771-463d-bc4a-7ca56563b6b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '548f411fb3a24eaea0043a7b3438584d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:50.357749 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-da0fc452-b771-463d-bc4a-7ca56563b6b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '548f411fb3a24eaea0043a7b3438584d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:50.358387 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-da0fc452-b771-463d-bc4a-7ca56563b6b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '548f411fb3a24eaea0043a7b3438584d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:50.359685 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:33:50.373033 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-da0fc452-b771-463d-bc4a-7ca56563b6b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '548f411fb3a24eaea0043a7b3438584d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:50.373774 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-da0fc452-b771-463d-bc4a-7ca56563b6b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '548f411fb3a24eaea0043a7b3438584d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:50.386979 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1171/2343] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:33:50 2022] DELETE /identity/v3/users/548f411fb3a24eaea0043a7b3438584d => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:33:50.672351 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25a6dcd-d806-486e-8373-38ef646458f8 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:50.673875 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a25a6dcd-d806-486e-8373-38ef646458f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:50.689282 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25a6dcd-d806-486e-8373-38ef646458f8 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:50.733937 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:33:50.751486 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c30ad-ddb6-4c42-ad7a-81293b61d74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:50.752156 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c30ad-ddb6-4c42-ad7a-81293b61d74a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:50.752438 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c30ad-ddb6-4c42-ad7a-81293b61d74a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:50.752712 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c30ad-ddb6-4c42-ad7a-81293b61d74a admin admin] PATH_INFO: `/v3/projects/0c0bbd6f6c9f47048911bfe3c1ba19f2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:50.753960 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33c30ad-ddb6-4c42-ad7a-81293b61d74a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c0bbd6f6c9f47048911bfe3c1ba19f2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:50.756189 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33c30ad-ddb6-4c42-ad7a-81293b61d74a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:50.806855 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a33c30ad-ddb6-4c42-ad7a-81293b61d74a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c0bbd6f6c9f47048911bfe3c1ba19f2', 'request_id': 'req-a33c30ad-ddb6-4c42-ad7a-81293b61d74a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:50.816111 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a33c30ad-ddb6-4c42-ad7a-81293b61d74a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c0bbd6f6c9f47048911bfe3c1ba19f2', 'request_id': 'req-a33c30ad-ddb6-4c42-ad7a-81293b61d74a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:50.822342 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a33c30ad-ddb6-4c42-ad7a-81293b61d74a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c0bbd6f6c9f47048911bfe3c1ba19f2', 'request_id': 'req-a33c30ad-ddb6-4c42-ad7a-81293b61d74a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:50.833795 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a33c30ad-ddb6-4c42-ad7a-81293b61d74a admin admin] The token cache is being invalidate because project 0c0bbd6f6c9f47048911bfe3c1ba19f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:33:50.834073 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a33c30ad-ddb6-4c42-ad7a-81293b61d74a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:50.834740 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a33c30ad-ddb6-4c42-ad7a-81293b61d74a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:50.836436 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1173/2344] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:33:50 2022] DELETE /identity/v3/projects/0c0bbd6f6c9f47048911bfe3c1ba19f2 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:33:50.841447 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6859978c-a283-427e-8a03-05746b82aec4 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:50.841983 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6859978c-a283-427e-8a03-05746b82aec4 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:50.841983 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6859978c-a283-427e-8a03-05746b82aec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:50.863922 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-6859978c-a283-427e-8a03-05746b82aec4 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:33:50.865141 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6859978c-a283-427e-8a03-05746b82aec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:50.929601 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1172/2345] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:33:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:33:50.938043 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35aeae-aa19-4dfa-8fea-0ea7ef754fec None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:50.938321 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35aeae-aa19-4dfa-8fea-0ea7ef754fec None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:50.938603 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35aeae-aa19-4dfa-8fea-0ea7ef754fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:50.962088 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1a35aeae-aa19-4dfa-8fea-0ea7ef754fec None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:33:50.963657 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1a35aeae-aa19-4dfa-8fea-0ea7ef754fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:50.972733 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1174/2346] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:33:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:33:50.978441 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6859978c-a283-427e-8a03-05746b82aec4 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:50.992070 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6859978c-a283-427e-8a03-05746b82aec4 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:50.997882 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da663517-7fde-4c60-a5ac-710c204cbb25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:50.998491 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da663517-7fde-4c60-a5ac-710c204cbb25 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:50.998766 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da663517-7fde-4c60-a5ac-710c204cbb25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:50.999061 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da663517-7fde-4c60-a5ac-710c204cbb25 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:50.999758 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da663517-7fde-4c60-a5ac-710c204cbb25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:33:51.000059 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da663517-7fde-4c60-a5ac-710c204cbb25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:51.001657 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da663517-7fde-4c60-a5ac-710c204cbb25 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:51.014230 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1173/2347] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:33:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:33:51.018596 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a35aeae-aa19-4dfa-8fea-0ea7ef754fec None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:51.028814 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a35aeae-aa19-4dfa-8fea-0ea7ef754fec None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:51.033695 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426c53e6-e8fa-449d-adc7-c1890f9760d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:51.034308 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c53e6-e8fa-449d-adc7-c1890f9760d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:51.034584 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c53e6-e8fa-449d-adc7-c1890f9760d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:51.035002 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c53e6-e8fa-449d-adc7-c1890f9760d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:51.035859 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426c53e6-e8fa-449d-adc7-c1890f9760d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:51.037767 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426c53e6-e8fa-449d-adc7-c1890f9760d3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:51.068600 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1175/2348] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:33:51 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:33:51.072885 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da663517-7fde-4c60-a5ac-710c204cbb25 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:51.083436 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da663517-7fde-4c60-a5ac-710c204cbb25 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:51.143951 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80635fc1-8922-4346-9556-50bffaf6a637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:51.144489 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80635fc1-8922-4346-9556-50bffaf6a637 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:51.144767 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80635fc1-8922-4346-9556-50bffaf6a637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:51.145102 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80635fc1-8922-4346-9556-50bffaf6a637 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:51.145763 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80635fc1-8922-4346-9556-50bffaf6a637 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:51.148289 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80635fc1-8922-4346-9556-50bffaf6a637 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:51.173820 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1174/2349] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:33:51 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:33:51.178800 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426c53e6-e8fa-449d-adc7-c1890f9760d3 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:51.189055 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426c53e6-e8fa-449d-adc7-c1890f9760d3 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:51.196272 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe49d33b-6685-4763-adcc-3800a89fc6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:51.197028 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49d33b-6685-4763-adcc-3800a89fc6ed admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:51.197365 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49d33b-6685-4763-adcc-3800a89fc6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:51.197652 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49d33b-6685-4763-adcc-3800a89fc6ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:51.198639 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe49d33b-6685-4763-adcc-3800a89fc6ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:51.200858 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe49d33b-6685-4763-adcc-3800a89fc6ed admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:51.215752 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1176/2350] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 10:33:51.221348 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80635fc1-8922-4346-9556-50bffaf6a637 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:51.232781 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80635fc1-8922-4346-9556-50bffaf6a637 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:51.238327 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11726227-698c-459e-9504-380032ee93cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:51.239252 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11726227-698c-459e-9504-380032ee93cc admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:51.239558 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11726227-698c-459e-9504-380032ee93cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:51.239827 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11726227-698c-459e-9504-380032ee93cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:51.240522 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11726227-698c-459e-9504-380032ee93cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:51.242106 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11726227-698c-459e-9504-380032ee93cc admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:51.253242 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1175/2351] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:33:51.257761 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe49d33b-6685-4763-adcc-3800a89fc6ed admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:51.268370 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe49d33b-6685-4763-adcc-3800a89fc6ed admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:51.275896 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88765094-51f6-42cf-93ef-2eaf6d29d220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:51.276724 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88765094-51f6-42cf-93ef-2eaf6d29d220 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:51.277008 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88765094-51f6-42cf-93ef-2eaf6d29d220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:51.277287 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88765094-51f6-42cf-93ef-2eaf6d29d220 admin admin] PATH_INFO: `/v3/projects/20649d548c8d47d19c899faff30998ab/users/433c82ab639e4e56807fc22876c0af5c/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:51.293267 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88765094-51f6-42cf-93ef-2eaf6d29d220 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20649d548c8d47d19c899faff30998ab, user_id=433c82ab639e4e56807fc22876c0af5c, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:51.295957 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88765094-51f6-42cf-93ef-2eaf6d29d220 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:51.311183 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1177/2352] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:33:51 2022] PUT /identity/v3/projects/20649d548c8d47d19c899faff30998ab/users/433c82ab639e4e56807fc22876c0af5c/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:33:51.381873 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f1545-ba72-4197-8f45-a9df2e54b518 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:51.382148 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f1545-ba72-4197-8f45-a9df2e54b518 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:51.382434 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f1545-ba72-4197-8f45-a9df2e54b518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:51.412595 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-1f1f1545-ba72-4197-8f45-a9df2e54b518 None None] MFA Rules not processed for user `433c82ab639e4e56807fc22876c0af5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:33:51.413728 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1f1f1545-ba72-4197-8f45-a9df2e54b518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:51.446873 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:33:51.490049 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1176/2353] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:33:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:33:51.502589 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88765094-51f6-42cf-93ef-2eaf6d29d220 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:51.504773 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-88765094-51f6-42cf-93ef-2eaf6d29d220 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:51.570801 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88765094-51f6-42cf-93ef-2eaf6d29d220 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:51.575197 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a0bc86-dadd-4b0b-a709-65a480803848 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:51.575811 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a0bc86-dadd-4b0b-a709-65a480803848 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:51.576090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a0bc86-dadd-4b0b-a709-65a480803848 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:51.576360 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a0bc86-dadd-4b0b-a709-65a480803848 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:51.585143 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a0bc86-dadd-4b0b-a709-65a480803848 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:51.586585 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a0bc86-dadd-4b0b-a709-65a480803848 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:51.595769 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1178/2354] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:33:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:33:51.616646 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f1545-ba72-4197-8f45-a9df2e54b518 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:51.682314 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f1545-ba72-4197-8f45-a9df2e54b518 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:51.687071 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7c2b4c-037a-4d8c-84d6-d9acbad209c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:51.687722 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c2b4c-037a-4d8c-84d6-d9acbad209c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:51.688004 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c2b4c-037a-4d8c-84d6-d9acbad209c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:51.688279 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c2b4c-037a-4d8c-84d6-d9acbad209c4 admin admin] PATH_INFO: `/v3/users/433c82ab639e4e56807fc22876c0af5c` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:51.689483 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7c2b4c-037a-4d8c-84d6-d9acbad209c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=433c82ab639e4e56807fc22876c0af5c)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:51.691451 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7c2b4c-037a-4d8c-84d6-d9acbad209c4 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:51.749065 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-bc7c2b4c-037a-4d8c-84d6-d9acbad209c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '433c82ab639e4e56807fc22876c0af5c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:51.758800 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-bc7c2b4c-037a-4d8c-84d6-d9acbad209c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '433c82ab639e4e56807fc22876c0af5c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:51.759617 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-bc7c2b4c-037a-4d8c-84d6-d9acbad209c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '433c82ab639e4e56807fc22876c0af5c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:51.771720 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-bc7c2b4c-037a-4d8c-84d6-d9acbad209c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '433c82ab639e4e56807fc22876c0af5c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:51.772718 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-bc7c2b4c-037a-4d8c-84d6-d9acbad209c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '433c82ab639e4e56807fc22876c0af5c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:51.786125 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1177/2355] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:33:51 2022] DELETE /identity/v3/users/433c82ab639e4e56807fc22876c0af5c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:33:51.790827 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cba2e5-8677-4c80-8d21-b90208e5c6c3 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:51.791345 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cba2e5-8677-4c80-8d21-b90208e5c6c3 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:51.791458 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cba2e5-8677-4c80-8d21-b90208e5c6c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:51.811476 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-87cba2e5-8677-4c80-8d21-b90208e5c6c3 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:33:51.812718 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-87cba2e5-8677-4c80-8d21-b90208e5c6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:51.821588 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:33:51.878370 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1179/2356] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:33:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:33:51.888414 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7c2b4c-037a-4d8c-84d6-d9acbad209c4 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:51.890035 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-bc7c2b4c-037a-4d8c-84d6-d9acbad209c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:51.947820 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:33:51.953930 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7c2b4c-037a-4d8c-84d6-d9acbad209c4 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:51.959325 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12306663-3214-4ce4-9826-2bbc7d09b3aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:51.959986 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12306663-3214-4ce4-9826-2bbc7d09b3aa service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:51.960322 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12306663-3214-4ce4-9826-2bbc7d09b3aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:51.960848 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12306663-3214-4ce4-9826-2bbc7d09b3aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:51.970645 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12306663-3214-4ce4-9826-2bbc7d09b3aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:51.972137 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12306663-3214-4ce4-9826-2bbc7d09b3aa service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:51.982565 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1178/2357] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:33:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:33:52.262511 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cba2e5-8677-4c80-8d21-b90208e5c6c3 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:52.264488 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-87cba2e5-8677-4c80-8d21-b90208e5c6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:52.283397 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cba2e5-8677-4c80-8d21-b90208e5c6c3 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:52.290110 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e9e263-fafd-45a1-844c-f3474fbbc57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:52.290852 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9e263-fafd-45a1-844c-f3474fbbc57d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:52.291106 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9e263-fafd-45a1-844c-f3474fbbc57d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:52.291377 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9e263-fafd-45a1-844c-f3474fbbc57d admin admin] PATH_INFO: `/v3/projects/20649d548c8d47d19c899faff30998ab` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:52.292549 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e9e263-fafd-45a1-844c-f3474fbbc57d admin admin] RBAC: Authorizing `identity:delete_project(project_id=20649d548c8d47d19c899faff30998ab)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:52.294455 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e9e263-fafd-45a1-844c-f3474fbbc57d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:52.347207 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f3e9e263-fafd-45a1-844c-f3474fbbc57d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20649d548c8d47d19c899faff30998ab', 'request_id': 'req-f3e9e263-fafd-45a1-844c-f3474fbbc57d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:52.356607 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f3e9e263-fafd-45a1-844c-f3474fbbc57d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20649d548c8d47d19c899faff30998ab', 'request_id': 'req-f3e9e263-fafd-45a1-844c-f3474fbbc57d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:52.363112 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f3e9e263-fafd-45a1-844c-f3474fbbc57d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20649d548c8d47d19c899faff30998ab', 'request_id': 'req-f3e9e263-fafd-45a1-844c-f3474fbbc57d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:52.374484 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f3e9e263-fafd-45a1-844c-f3474fbbc57d admin admin] The token cache is being invalidate because project 20649d548c8d47d19c899faff30998ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:33:52.374770 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f3e9e263-fafd-45a1-844c-f3474fbbc57d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:52.375475 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f3e9e263-fafd-45a1-844c-f3474fbbc57d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:33:52.377039 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1180/2358] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:33:52 2022] DELETE /identity/v3/projects/20649d548c8d47d19c899faff30998ab => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:33:52.382251 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de21313-51d9-4a91-b943-ab7649a75524 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:52.382544 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de21313-51d9-4a91-b943-ab7649a75524 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:52.382858 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de21313-51d9-4a91-b943-ab7649a75524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:52.403304 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-6de21313-51d9-4a91-b943-ab7649a75524 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:33:52.404510 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6de21313-51d9-4a91-b943-ab7649a75524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:52.471134 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1179/2359] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:33:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:33:52.482877 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604feda2-4894-47ab-b652-ed41e2bbf5e7 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:52.483404 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604feda2-4894-47ab-b652-ed41e2bbf5e7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:52.483857 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604feda2-4894-47ab-b652-ed41e2bbf5e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:52.507258 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-604feda2-4894-47ab-b652-ed41e2bbf5e7 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:33:52.508471 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-604feda2-4894-47ab-b652-ed41e2bbf5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:52.518033 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1181/2360] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:33:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:33:52.523746 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de21313-51d9-4a91-b943-ab7649a75524 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:52.534568 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de21313-51d9-4a91-b943-ab7649a75524 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:52.540719 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96463ce9-a153-44ca-9024-c778ba1dec4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:52.541340 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96463ce9-a153-44ca-9024-c778ba1dec4a admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:52.541615 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96463ce9-a153-44ca-9024-c778ba1dec4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:52.541883 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96463ce9-a153-44ca-9024-c778ba1dec4a admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:52.542576 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96463ce9-a153-44ca-9024-c778ba1dec4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:33:52.542880 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96463ce9-a153-44ca-9024-c778ba1dec4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:52.544506 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96463ce9-a153-44ca-9024-c778ba1dec4a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:52.558098 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1180/2361] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:33:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:33:52.562694 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604feda2-4894-47ab-b652-ed41e2bbf5e7 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:52.572751 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604feda2-4894-47ab-b652-ed41e2bbf5e7 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:52.577780 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c47e1f-b3d3-4f22-becf-ee815a67f314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:52.578390 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c47e1f-b3d3-4f22-becf-ee815a67f314 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:52.578668 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c47e1f-b3d3-4f22-becf-ee815a67f314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:52.578960 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c47e1f-b3d3-4f22-becf-ee815a67f314 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:52.579628 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c47e1f-b3d3-4f22-becf-ee815a67f314 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:52.581531 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c47e1f-b3d3-4f22-becf-ee815a67f314 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:52.612225 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1182/2362] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:33:52 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:33:52.616621 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96463ce9-a153-44ca-9024-c778ba1dec4a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:52.626661 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96463ce9-a153-44ca-9024-c778ba1dec4a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:52.686884 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b368c43-0549-45e0-9530-25341a82758e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:52.687517 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b368c43-0549-45e0-9530-25341a82758e admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:52.687804 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b368c43-0549-45e0-9530-25341a82758e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:52.688062 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b368c43-0549-45e0-9530-25341a82758e admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:52.688721 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b368c43-0549-45e0-9530-25341a82758e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:52.690676 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b368c43-0549-45e0-9530-25341a82758e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:52.713774 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1181/2363] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:33:52 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:33:52.718172 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c47e1f-b3d3-4f22-becf-ee815a67f314 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:52.729380 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c47e1f-b3d3-4f22-becf-ee815a67f314 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:52.736783 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2bd6d-8e02-46e2-9690-1c178b420e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:52.737644 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2bd6d-8e02-46e2-9690-1c178b420e9d admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:52.737919 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2bd6d-8e02-46e2-9690-1c178b420e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:52.738221 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2bd6d-8e02-46e2-9690-1c178b420e9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:52.739920 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad2bd6d-8e02-46e2-9690-1c178b420e9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:52.742334 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad2bd6d-8e02-46e2-9690-1c178b420e9d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:52.753552 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1183/2364] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:33:52.757847 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b368c43-0549-45e0-9530-25341a82758e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:52.768404 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b368c43-0549-45e0-9530-25341a82758e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:52.773862 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a6ed5-d542-4f0e-9f5e-a2837abda8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:52.774507 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a6ed5-d542-4f0e-9f5e-a2837abda8ea admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:52.774787 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a6ed5-d542-4f0e-9f5e-a2837abda8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:52.775119 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a6ed5-d542-4f0e-9f5e-a2837abda8ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:52.775812 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193a6ed5-d542-4f0e-9f5e-a2837abda8ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:52.777479 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193a6ed5-d542-4f0e-9f5e-a2837abda8ea admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:52.789266 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1182/2365] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:33:52.793490 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2bd6d-8e02-46e2-9690-1c178b420e9d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:52.804217 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2bd6d-8e02-46e2-9690-1c178b420e9d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:52.809562 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6829af-fbf9-429e-8264-dad9c916a1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:52.810226 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6829af-fbf9-429e-8264-dad9c916a1a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:52.810497 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6829af-fbf9-429e-8264-dad9c916a1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:52.810763 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6829af-fbf9-429e-8264-dad9c916a1a0 admin admin] PATH_INFO: `/v3/projects/b40029ca79ea43e79882c57b593d1507/users/3f7b7f9f9ec945e1b8511d369c43da29/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:52.826369 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6829af-fbf9-429e-8264-dad9c916a1a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b40029ca79ea43e79882c57b593d1507, user_id=3f7b7f9f9ec945e1b8511d369c43da29, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:52.829048 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6829af-fbf9-429e-8264-dad9c916a1a0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:52.845367 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1184/2366] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:33:52 2022] PUT /identity/v3/projects/b40029ca79ea43e79882c57b593d1507/users/3f7b7f9f9ec945e1b8511d369c43da29/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:33:52.857673 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3661fb5-e9c6-4901-9d71-d0a94ecb4a34 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:52.857919 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3661fb5-e9c6-4901-9d71-d0a94ecb4a34 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:52.858252 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3661fb5-e9c6-4901-9d71-d0a94ecb4a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:52.889248 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-d3661fb5-e9c6-4901-9d71-d0a94ecb4a34 None None] MFA Rules not processed for user `3f7b7f9f9ec945e1b8511d369c43da29`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:33:52.890497 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d3661fb5-e9c6-4901-9d71-d0a94ecb4a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:52.938023 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:33:52.960843 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1183/2367] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:33:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:33:52.965620 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6829af-fbf9-429e-8264-dad9c916a1a0 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:53.000558 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:33:53.032592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6829af-fbf9-429e-8264-dad9c916a1a0 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:53.038515 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e607c6f-f84b-45c6-9d3a-3586c3cee520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:53.039158 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e607c6f-f84b-45c6-9d3a-3586c3cee520 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:53.039436 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e607c6f-f84b-45c6-9d3a-3586c3cee520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:53.039703 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e607c6f-f84b-45c6-9d3a-3586c3cee520 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:53.040361 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e607c6f-f84b-45c6-9d3a-3586c3cee520 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:53.042288 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e607c6f-f84b-45c6-9d3a-3586c3cee520 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:53.073223 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1185/2368] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:33:52 2022] POST /identity/v3/projects => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:33:53.079516 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3661fb5-e9c6-4901-9d71-d0a94ecb4a34 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:53.091862 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3661fb5-e9c6-4901-9d71-d0a94ecb4a34 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:53.149427 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2421449-d2e2-46fd-89a0-a898048e406d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:53.150042 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2421449-d2e2-46fd-89a0-a898048e406d admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:53.150346 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2421449-d2e2-46fd-89a0-a898048e406d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:53.150615 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2421449-d2e2-46fd-89a0-a898048e406d admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:53.151295 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2421449-d2e2-46fd-89a0-a898048e406d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:53.153214 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2421449-d2e2-46fd-89a0-a898048e406d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:53.178043 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1184/2369] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:33:53 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:33:53.182243 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e607c6f-f84b-45c6-9d3a-3586c3cee520 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:53.192781 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e607c6f-f84b-45c6-9d3a-3586c3cee520 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:53.197679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb9700f-b32a-42f9-b110-85079d71571a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:53.198289 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb9700f-b32a-42f9-b110-85079d71571a admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:53.198558 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb9700f-b32a-42f9-b110-85079d71571a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:53.198828 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb9700f-b32a-42f9-b110-85079d71571a admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:53.199541 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb9700f-b32a-42f9-b110-85079d71571a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:53.201135 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb9700f-b32a-42f9-b110-85079d71571a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:53.211930 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1186/2370] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:33:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:33:53.215999 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2421449-d2e2-46fd-89a0-a898048e406d admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:53.226379 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2421449-d2e2-46fd-89a0-a898048e406d admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:53.232486 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dac71e-319f-4192-8289-f609d15874f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:53.233086 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dac71e-319f-4192-8289-f609d15874f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:53.233367 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dac71e-319f-4192-8289-f609d15874f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:53.233633 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dac71e-319f-4192-8289-f609d15874f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:53.234316 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dac71e-319f-4192-8289-f609d15874f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:53.235940 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dac71e-319f-4192-8289-f609d15874f3 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:53.246435 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1185/2371] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:33:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 10:33:53.250451 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb9700f-b32a-42f9-b110-85079d71571a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:53.260673 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb9700f-b32a-42f9-b110-85079d71571a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:53.266286 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c25a5-3e79-4b3a-87d8-362b2fdc4de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:53.266984 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c25a5-3e79-4b3a-87d8-362b2fdc4de5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:53.267255 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c25a5-3e79-4b3a-87d8-362b2fdc4de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:53.267520 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c25a5-3e79-4b3a-87d8-362b2fdc4de5 admin admin] PATH_INFO: `/v3/projects/aafc599d074c43f38d43316abb5d90a0/users/f2ed64a8153a46728d148cfeb095438b/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:53.283335 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61c25a5-3e79-4b3a-87d8-362b2fdc4de5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aafc599d074c43f38d43316abb5d90a0, user_id=f2ed64a8153a46728d148cfeb095438b, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:53.286064 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61c25a5-3e79-4b3a-87d8-362b2fdc4de5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:53.300894 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1187/2372] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:33:53 2022] PUT /identity/v3/projects/aafc599d074c43f38d43316abb5d90a0/users/f2ed64a8153a46728d148cfeb095438b/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:33:53.312729 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6340ee2-ea15-405a-876d-f67616cd3feb None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:53.313049 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6340ee2-ea15-405a-876d-f67616cd3feb None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:53.313348 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6340ee2-ea15-405a-876d-f67616cd3feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:53.342713 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b6340ee2-ea15-405a-876d-f67616cd3feb None None] MFA Rules not processed for user `f2ed64a8153a46728d148cfeb095438b`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:33:53.344061 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b6340ee2-ea15-405a-876d-f67616cd3feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:53.411846 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1186/2373] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:33:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:33:53.416593 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c25a5-3e79-4b3a-87d8-362b2fdc4de5 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:53.483044 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c25a5-3e79-4b3a-87d8-362b2fdc4de5 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:53.488443 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eae89d3-6aed-4534-9f73-78110d940b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:53.489082 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae89d3-6aed-4534-9f73-78110d940b5c admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:53.489402 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae89d3-6aed-4534-9f73-78110d940b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:53.489671 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae89d3-6aed-4534-9f73-78110d940b5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:53.490329 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eae89d3-6aed-4534-9f73-78110d940b5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:53.492274 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eae89d3-6aed-4534-9f73-78110d940b5c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:53.525342 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1188/2374] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:33:53 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:33:53.531032 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6340ee2-ea15-405a-876d-f67616cd3feb None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:53.542740 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6340ee2-ea15-405a-876d-f67616cd3feb None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:53.595823 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:33:53.606176 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff405714-9a09-4afc-a556-57aa90f310b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:53.606810 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff405714-9a09-4afc-a556-57aa90f310b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:53.607137 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff405714-9a09-4afc-a556-57aa90f310b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:53.607413 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff405714-9a09-4afc-a556-57aa90f310b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:53.608086 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff405714-9a09-4afc-a556-57aa90f310b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:53.610073 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff405714-9a09-4afc-a556-57aa90f310b7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:53.633712 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1187/2375] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:33:53 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:33:53.637782 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eae89d3-6aed-4534-9f73-78110d940b5c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:53.649420 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eae89d3-6aed-4534-9f73-78110d940b5c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:53.655202 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0510f5-3182-4659-84b3-0c8ac6e29104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:53.655819 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0510f5-3182-4659-84b3-0c8ac6e29104 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:53.656094 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0510f5-3182-4659-84b3-0c8ac6e29104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:53.656367 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0510f5-3182-4659-84b3-0c8ac6e29104 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:53.657058 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0510f5-3182-4659-84b3-0c8ac6e29104 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:53.658665 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0510f5-3182-4659-84b3-0c8ac6e29104 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:53.670232 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1189/2376] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:33:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 10:33:53.674488 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff405714-9a09-4afc-a556-57aa90f310b7 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:53.684872 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff405714-9a09-4afc-a556-57aa90f310b7 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:53.689869 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e963400b-6f31-4372-9f33-2481bc1fa3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:53.690530 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963400b-6f31-4372-9f33-2481bc1fa3a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:53.690814 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963400b-6f31-4372-9f33-2481bc1fa3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:53.691113 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963400b-6f31-4372-9f33-2481bc1fa3a7 admin admin] PATH_INFO: `/v3/projects/704f6bcd6a554309904e1cdc0be53098/users/d6ec95adfd994426ac30e4afed7a3bd8/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:53.706530 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e963400b-6f31-4372-9f33-2481bc1fa3a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=704f6bcd6a554309904e1cdc0be53098, user_id=d6ec95adfd994426ac30e4afed7a3bd8, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:53.709234 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e963400b-6f31-4372-9f33-2481bc1fa3a7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:53.723991 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1188/2377] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:33:53 2022] PUT /identity/v3/projects/704f6bcd6a554309904e1cdc0be53098/users/d6ec95adfd994426ac30e4afed7a3bd8/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:33:53.729811 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0510f5-3182-4659-84b3-0c8ac6e29104 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:53.745852 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0510f5-3182-4659-84b3-0c8ac6e29104 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:53.802989 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:33:53.810566 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16eb1ce-a335-4be6-85d6-2957446bbd03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:53.811415 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16eb1ce-a335-4be6-85d6-2957446bbd03 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:53.811953 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16eb1ce-a335-4be6-85d6-2957446bbd03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:53.812237 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16eb1ce-a335-4be6-85d6-2957446bbd03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:53.812957 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16eb1ce-a335-4be6-85d6-2957446bbd03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:53.814626 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16eb1ce-a335-4be6-85d6-2957446bbd03 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:53.824828 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1190/2378] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:33:53 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:33:53.829107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e963400b-6f31-4372-9f33-2481bc1fa3a7 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:53.839657 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e963400b-6f31-4372-9f33-2481bc1fa3a7 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:53.844864 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651e9385-49e6-419f-9163-a3aa4e1dbea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:53.845569 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e9385-49e6-419f-9163-a3aa4e1dbea9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:53.845847 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e9385-49e6-419f-9163-a3aa4e1dbea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:53.846090 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e9385-49e6-419f-9163-a3aa4e1dbea9 admin admin] PATH_INFO: `/v3/projects/704f6bcd6a554309904e1cdc0be53098/users/d6ec95adfd994426ac30e4afed7a3bd8/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:53.851485 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651e9385-49e6-419f-9163-a3aa4e1dbea9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=704f6bcd6a554309904e1cdc0be53098, user_id=d6ec95adfd994426ac30e4afed7a3bd8, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:53.851485 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651e9385-49e6-419f-9163-a3aa4e1dbea9 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:53.862116 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1189/2379] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:33:53 2022] PUT /identity/v3/projects/704f6bcd6a554309904e1cdc0be53098/users/d6ec95adfd994426ac30e4afed7a3bd8/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:33:53.872821 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda54e16-9d52-4591-aca1-95b4c46786c0 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:53.873106 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda54e16-9d52-4591-aca1-95b4c46786c0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:53.873428 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda54e16-9d52-4591-aca1-95b4c46786c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:53.905529 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-bda54e16-9d52-4591-aca1-95b4c46786c0 None None] MFA Rules not processed for user `d6ec95adfd994426ac30e4afed7a3bd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:33:53.906729 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bda54e16-9d52-4591-aca1-95b4c46786c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:53.978679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1191/2380] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:33:53 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:33:53.990677 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651e9385-49e6-419f-9163-a3aa4e1dbea9 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:53.992755 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-651e9385-49e6-419f-9163-a3aa4e1dbea9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:54.056294 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651e9385-49e6-419f-9163-a3aa4e1dbea9 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:54.060038 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a476e80-78c7-4074-842d-9c3ce86ce590 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:54.060795 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a476e80-78c7-4074-842d-9c3ce86ce590 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:54.061081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a476e80-78c7-4074-842d-9c3ce86ce590 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:54.061355 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a476e80-78c7-4074-842d-9c3ce86ce590 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:54.121042 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a476e80-78c7-4074-842d-9c3ce86ce590 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:54.122491 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a476e80-78c7-4074-842d-9c3ce86ce590 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:54.131918 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1190/2381] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:33:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:33:56.721181 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda54e16-9d52-4591-aca1-95b4c46786c0 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:56.722830 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bda54e16-9d52-4591-aca1-95b4c46786c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:33:56.737077 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda54e16-9d52-4591-aca1-95b4c46786c0 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:56.741984 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506dd068-6554-4075-b7e0-30cb0212ec81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:56.742606 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506dd068-6554-4075-b7e0-30cb0212ec81 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:56.742888 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506dd068-6554-4075-b7e0-30cb0212ec81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:56.743166 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506dd068-6554-4075-b7e0-30cb0212ec81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:56.804851 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506dd068-6554-4075-b7e0-30cb0212ec81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:56.806228 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506dd068-6554-4075-b7e0-30cb0212ec81 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:56.815670 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1192/2382] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:33:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:33:57.377553 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a476e80-78c7-4074-842d-9c3ce86ce590 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:33:57.389051 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a476e80-78c7-4074-842d-9c3ce86ce590 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:33:57.395876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0eda02-ccc4-4cbc-96f4-b73d5d917116 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:33:57.396778 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0eda02-ccc4-4cbc-96f4-b73d5d917116 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:33:57.397451 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0eda02-ccc4-4cbc-96f4-b73d5d917116 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:33:57.397728 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0eda02-ccc4-4cbc-96f4-b73d5d917116 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:33:57.412555 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0eda02-ccc4-4cbc-96f4-b73d5d917116 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:33:57.414413 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0eda02-ccc4-4cbc-96f4-b73d5d917116 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:33:57.424237 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1191/2383] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:33:57 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:34:01.355511 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506dd068-6554-4075-b7e0-30cb0212ec81 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:34:01.422295 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506dd068-6554-4075-b7e0-30cb0212ec81 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:34:01.427992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c531eba1-aa96-4ce1-80a2-ced2cda92c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:34:01.428635 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531eba1-aa96-4ce1-80a2-ced2cda92c45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:34:01.428912 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531eba1-aa96-4ce1-80a2-ced2cda92c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:34:01.429184 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531eba1-aa96-4ce1-80a2-ced2cda92c45 admin admin] PATH_INFO: `/v3/users/3f7b7f9f9ec945e1b8511d369c43da29` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:34:01.430506 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c531eba1-aa96-4ce1-80a2-ced2cda92c45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f7b7f9f9ec945e1b8511d369c43da29)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:34:01.432499 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c531eba1-aa96-4ce1-80a2-ced2cda92c45 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:34:01.453250 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:34:01.492207 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c531eba1-aa96-4ce1-80a2-ced2cda92c45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f7b7f9f9ec945e1b8511d369c43da29'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:34:01.492896 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c531eba1-aa96-4ce1-80a2-ced2cda92c45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f7b7f9f9ec945e1b8511d369c43da29'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:34:01.493664 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c531eba1-aa96-4ce1-80a2-ced2cda92c45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f7b7f9f9ec945e1b8511d369c43da29'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:34:01.504516 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c531eba1-aa96-4ce1-80a2-ced2cda92c45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f7b7f9f9ec945e1b8511d369c43da29'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:34:01.512103 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c531eba1-aa96-4ce1-80a2-ced2cda92c45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f7b7f9f9ec945e1b8511d369c43da29'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:34:01.530716 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1193/2384] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:34:01 2022] DELETE /identity/v3/users/3f7b7f9f9ec945e1b8511d369c43da29 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:34:01.535349 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a0fbfe-8a99-43e5-9703-df691edcdb27 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:34:01.535816 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a0fbfe-8a99-43e5-9703-df691edcdb27 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:34:01.535931 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a0fbfe-8a99-43e5-9703-df691edcdb27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:34:01.555610 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-f2a0fbfe-8a99-43e5-9703-df691edcdb27 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:34:01.556813 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f2a0fbfe-8a99-43e5-9703-df691edcdb27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:34:01.607311 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:34:01.628514 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1192/2385] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:34:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:34:01.638535 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c531eba1-aa96-4ce1-80a2-ced2cda92c45 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:34:01.640202 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c531eba1-aa96-4ce1-80a2-ced2cda92c45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:34:01.703261 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c531eba1-aa96-4ce1-80a2-ced2cda92c45 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:34:01.707258 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe2a37-3ab1-4146-a750-04e822a52152 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:34:01.707886 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe2a37-3ab1-4146-a750-04e822a52152 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:34:01.708171 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe2a37-3ab1-4146-a750-04e822a52152 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:34:01.708452 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe2a37-3ab1-4146-a750-04e822a52152 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:34:01.718893 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fe2a37-3ab1-4146-a750-04e822a52152 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:34:01.720383 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fe2a37-3ab1-4146-a750-04e822a52152 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:34:01.730014 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1194/2386] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:34:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:34:02.018590 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a0fbfe-8a99-43e5-9703-df691edcdb27 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:34:02.020205 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f2a0fbfe-8a99-43e5-9703-df691edcdb27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:34:02.033519 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a0fbfe-8a99-43e5-9703-df691edcdb27 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:34:02.038756 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b060c93-be0e-4a7f-ab79-cb6b207dc70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:34:02.040105 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b060c93-be0e-4a7f-ab79-cb6b207dc70e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:34:02.040389 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b060c93-be0e-4a7f-ab79-cb6b207dc70e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:34:02.040702 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b060c93-be0e-4a7f-ab79-cb6b207dc70e admin admin] PATH_INFO: `/v3/projects/b40029ca79ea43e79882c57b593d1507` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:34:02.041881 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b060c93-be0e-4a7f-ab79-cb6b207dc70e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b40029ca79ea43e79882c57b593d1507)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:34:02.043868 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b060c93-be0e-4a7f-ab79-cb6b207dc70e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:34:02.095263 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5b060c93-be0e-4a7f-ab79-cb6b207dc70e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b40029ca79ea43e79882c57b593d1507', 'request_id': 'req-5b060c93-be0e-4a7f-ab79-cb6b207dc70e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:34:02.101043 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5b060c93-be0e-4a7f-ab79-cb6b207dc70e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b40029ca79ea43e79882c57b593d1507', 'request_id': 'req-5b060c93-be0e-4a7f-ab79-cb6b207dc70e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:34:02.110208 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5b060c93-be0e-4a7f-ab79-cb6b207dc70e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b40029ca79ea43e79882c57b593d1507', 'request_id': 'req-5b060c93-be0e-4a7f-ab79-cb6b207dc70e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:34:02.121764 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5b060c93-be0e-4a7f-ab79-cb6b207dc70e admin admin] The token cache is being invalidate because project b40029ca79ea43e79882c57b593d1507 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:34:02.122098 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5b060c93-be0e-4a7f-ab79-cb6b207dc70e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:34:02.122785 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5b060c93-be0e-4a7f-ab79-cb6b207dc70e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:34:02.124436 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1193/2387] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:34:02 2022] DELETE /identity/v3/projects/b40029ca79ea43e79882c57b593d1507 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:34:02.128577 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe2a37-3ab1-4146-a750-04e822a52152 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:34:02.138874 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe2a37-3ab1-4146-a750-04e822a52152 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:34:02.205525 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece5e576-a1c7-4611-b7ee-b62f18193937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:34:02.206172 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece5e576-a1c7-4611-b7ee-b62f18193937 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:34:02.206453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece5e576-a1c7-4611-b7ee-b62f18193937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:34:02.206722 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece5e576-a1c7-4611-b7ee-b62f18193937 admin admin] PATH_INFO: `/v3/users/f2ed64a8153a46728d148cfeb095438b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:34:02.207904 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece5e576-a1c7-4611-b7ee-b62f18193937 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2ed64a8153a46728d148cfeb095438b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:34:02.209830 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece5e576-a1c7-4611-b7ee-b62f18193937 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:34:02.254554 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:34:02.260961 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ece5e576-a1c7-4611-b7ee-b62f18193937 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2ed64a8153a46728d148cfeb095438b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:34:02.261952 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ece5e576-a1c7-4611-b7ee-b62f18193937 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2ed64a8153a46728d148cfeb095438b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:34:02.263101 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ece5e576-a1c7-4611-b7ee-b62f18193937 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2ed64a8153a46728d148cfeb095438b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:34:02.272266 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ece5e576-a1c7-4611-b7ee-b62f18193937 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2ed64a8153a46728d148cfeb095438b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:34:02.280035 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ece5e576-a1c7-4611-b7ee-b62f18193937 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2ed64a8153a46728d148cfeb095438b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:34:02.299089 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1195/2388] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:34:02 2022] DELETE /identity/v3/users/f2ed64a8153a46728d148cfeb095438b => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:34:02.592598 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b060c93-be0e-4a7f-ab79-cb6b207dc70e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:34:02.594145 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5b060c93-be0e-4a7f-ab79-cb6b207dc70e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:34:02.607783 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b060c93-be0e-4a7f-ab79-cb6b207dc70e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:34:02.671022 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62248acc-6671-4183-83a4-9d4009955dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:34:02.671597 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62248acc-6671-4183-83a4-9d4009955dce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:34:02.671880 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62248acc-6671-4183-83a4-9d4009955dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:34:02.672153 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62248acc-6671-4183-83a4-9d4009955dce admin admin] PATH_INFO: `/v3/projects/aafc599d074c43f38d43316abb5d90a0` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:34:02.673295 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62248acc-6671-4183-83a4-9d4009955dce admin admin] RBAC: Authorizing `identity:delete_project(project_id=aafc599d074c43f38d43316abb5d90a0)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:34:02.675254 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62248acc-6671-4183-83a4-9d4009955dce admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:34:02.728292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-62248acc-6671-4183-83a4-9d4009955dce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aafc599d074c43f38d43316abb5d90a0', 'request_id': 'req-62248acc-6671-4183-83a4-9d4009955dce'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:34:02.733951 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-62248acc-6671-4183-83a4-9d4009955dce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aafc599d074c43f38d43316abb5d90a0', 'request_id': 'req-62248acc-6671-4183-83a4-9d4009955dce'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:34:02.742452 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-62248acc-6671-4183-83a4-9d4009955dce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aafc599d074c43f38d43316abb5d90a0', 'request_id': 'req-62248acc-6671-4183-83a4-9d4009955dce'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:34:02.750951 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:34:02.755694 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-62248acc-6671-4183-83a4-9d4009955dce admin admin] The token cache is being invalidate because project aafc599d074c43f38d43316abb5d90a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:34:02.756432 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-62248acc-6671-4183-83a4-9d4009955dce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:34:02.757321 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-62248acc-6671-4183-83a4-9d4009955dce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:34:02.760130 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1194/2389] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:34:02 2022] DELETE /identity/v3/projects/aafc599d074c43f38d43316abb5d90a0 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:34:02.764294 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece5e576-a1c7-4611-b7ee-b62f18193937 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:34:02.765795 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ece5e576-a1c7-4611-b7ee-b62f18193937 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:34:02.780252 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece5e576-a1c7-4611-b7ee-b62f18193937 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:34:02.838857 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef19e19-4069-4c1a-b5d5-15879d322134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:34:02.839535 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef19e19-4069-4c1a-b5d5-15879d322134 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:34:02.839816 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef19e19-4069-4c1a-b5d5-15879d322134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:34:02.840084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef19e19-4069-4c1a-b5d5-15879d322134 admin admin] PATH_INFO: `/v3/users/d6ec95adfd994426ac30e4afed7a3bd8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:34:02.841333 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef19e19-4069-4c1a-b5d5-15879d322134 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6ec95adfd994426ac30e4afed7a3bd8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:34:02.843324 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef19e19-4069-4c1a-b5d5-15879d322134 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:34:02.895992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9ef19e19-4069-4c1a-b5d5-15879d322134 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6ec95adfd994426ac30e4afed7a3bd8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:34:02.897027 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9ef19e19-4069-4c1a-b5d5-15879d322134 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6ec95adfd994426ac30e4afed7a3bd8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:34:02.897628 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9ef19e19-4069-4c1a-b5d5-15879d322134 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6ec95adfd994426ac30e4afed7a3bd8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:34:02.906577 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9ef19e19-4069-4c1a-b5d5-15879d322134 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6ec95adfd994426ac30e4afed7a3bd8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:34:02.920012 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9ef19e19-4069-4c1a-b5d5-15879d322134 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6ec95adfd994426ac30e4afed7a3bd8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:34:02.939906 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1196/2390] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:34:02 2022] DELETE /identity/v3/users/d6ec95adfd994426ac30e4afed7a3bd8 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:34:03.236021 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62248acc-6671-4183-83a4-9d4009955dce admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:34:03.237753 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-62248acc-6671-4183-83a4-9d4009955dce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:34:03.253027 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62248acc-6671-4183-83a4-9d4009955dce admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:34:03.315764 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f967a-c008-4211-98fc-d95a19f4ff2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:34:03.316433 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f967a-c008-4211-98fc-d95a19f4ff2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:34:03.316744 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f967a-c008-4211-98fc-d95a19f4ff2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:34:03.317006 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f967a-c008-4211-98fc-d95a19f4ff2e admin admin] PATH_INFO: `/v3/projects/704f6bcd6a554309904e1cdc0be53098` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:34:03.318413 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3f967a-c008-4211-98fc-d95a19f4ff2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=704f6bcd6a554309904e1cdc0be53098)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:34:03.320362 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3f967a-c008-4211-98fc-d95a19f4ff2e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:34:03.370281 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-be3f967a-c008-4211-98fc-d95a19f4ff2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '704f6bcd6a554309904e1cdc0be53098', 'request_id': 'req-be3f967a-c008-4211-98fc-d95a19f4ff2e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:34:03.376034 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-be3f967a-c008-4211-98fc-d95a19f4ff2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '704f6bcd6a554309904e1cdc0be53098', 'request_id': 'req-be3f967a-c008-4211-98fc-d95a19f4ff2e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:34:03.384606 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-be3f967a-c008-4211-98fc-d95a19f4ff2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '704f6bcd6a554309904e1cdc0be53098', 'request_id': 'req-be3f967a-c008-4211-98fc-d95a19f4ff2e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:34:03.396417 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-be3f967a-c008-4211-98fc-d95a19f4ff2e admin admin] The token cache is being invalidate because project 704f6bcd6a554309904e1cdc0be53098 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:34:03.397097 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-be3f967a-c008-4211-98fc-d95a19f4ff2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:34:03.397771 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-be3f967a-c008-4211-98fc-d95a19f4ff2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:34:03.399555 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1195/2391] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:34:03 2022] DELETE /identity/v3/projects/704f6bcd6a554309904e1cdc0be53098 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:34:03.404592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ad15ca-7185-41a0-8d5f-46da8e0b452e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:34:03.404952 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ad15ca-7185-41a0-8d5f-46da8e0b452e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:34:03.405327 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ad15ca-7185-41a0-8d5f-46da8e0b452e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:34:03.425850 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-10ad15ca-7185-41a0-8d5f-46da8e0b452e None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:34:03.427089 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-10ad15ca-7185-41a0-8d5f-46da8e0b452e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:34:03.460795 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:34:03.491347 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1197/2392] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:34:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:34:03.500379 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d79b46-8d0e-4565-a5be-b504008efc3c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:34:03.500662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d79b46-8d0e-4565-a5be-b504008efc3c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:34:03.500933 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d79b46-8d0e-4565-a5be-b504008efc3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:34:03.520201 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-94d79b46-8d0e-4565-a5be-b504008efc3c None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:34:03.521389 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-94d79b46-8d0e-4565-a5be-b504008efc3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:34:03.530299 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1196/2393] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:34:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:34:03.534562 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ad15ca-7185-41a0-8d5f-46da8e0b452e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:34:03.544782 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ad15ca-7185-41a0-8d5f-46da8e0b452e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:34:03.550082 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f312b7aa-59b4-4a9d-b978-aa80a2bbe38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:34:03.550706 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f312b7aa-59b4-4a9d-b978-aa80a2bbe38f admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:34:03.551013 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f312b7aa-59b4-4a9d-b978-aa80a2bbe38f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:34:03.551285 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f312b7aa-59b4-4a9d-b978-aa80a2bbe38f admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:34:03.552000 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f312b7aa-59b4-4a9d-b978-aa80a2bbe38f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:34:03.552505 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f312b7aa-59b4-4a9d-b978-aa80a2bbe38f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:34:03.553872 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f312b7aa-59b4-4a9d-b978-aa80a2bbe38f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:34:03.566167 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1198/2394] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:34:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:34:03.570824 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d79b46-8d0e-4565-a5be-b504008efc3c None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:34:03.580351 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d79b46-8d0e-4565-a5be-b504008efc3c None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:34:03.585455 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f2220-ad9d-4200-8c51-4b5c93506b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:34:03.586405 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f2220-ad9d-4200-8c51-4b5c93506b7a admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:34:03.586800 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f2220-ad9d-4200-8c51-4b5c93506b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:34:03.587111 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f2220-ad9d-4200-8c51-4b5c93506b7a admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:34:03.587759 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0f2220-ad9d-4200-8c51-4b5c93506b7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:34:03.589692 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0f2220-ad9d-4200-8c51-4b5c93506b7a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:34:03.629750 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1197/2395] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:34:03 2022] POST /identity/v3/projects => generated 367 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:34:03.631353 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f312b7aa-59b4-4a9d-b978-aa80a2bbe38f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:34:03.641906 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f312b7aa-59b4-4a9d-b978-aa80a2bbe38f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:34:03.707369 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7588b6-99c9-4c4e-ba0d-7e79c0719e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:34:03.707977 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7588b6-99c9-4c4e-ba0d-7e79c0719e1d admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:34:03.708282 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7588b6-99c9-4c4e-ba0d-7e79c0719e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:34:03.708533 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7588b6-99c9-4c4e-ba0d-7e79c0719e1d admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:34:03.709770 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7588b6-99c9-4c4e-ba0d-7e79c0719e1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:34:03.711724 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7588b6-99c9-4c4e-ba0d-7e79c0719e1d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:34:03.738132 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1199/2396] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:34:03 2022] POST /identity/v3/users => generated 284 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:34:03.742413 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f2220-ad9d-4200-8c51-4b5c93506b7a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:34:03.753324 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f2220-ad9d-4200-8c51-4b5c93506b7a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:34:03.758633 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1c02d-2be2-48f8-969b-c0625a4e4cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:34:03.759287 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1c02d-2be2-48f8-969b-c0625a4e4cbf admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:34:03.759573 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1c02d-2be2-48f8-969b-c0625a4e4cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:34:03.759843 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1c02d-2be2-48f8-969b-c0625a4e4cbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:34:03.760541 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d1c02d-2be2-48f8-969b-c0625a4e4cbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:34:03.762146 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d1c02d-2be2-48f8-969b-c0625a4e4cbf admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:34:03.773035 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1198/2397] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:34:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:34:03.777515 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7588b6-99c9-4c4e-ba0d-7e79c0719e1d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:34:03.788301 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7588b6-99c9-4c4e-ba0d-7e79c0719e1d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:34:03.793236 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec112184-f961-42e1-9a9a-129abd1d7341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:34:03.793955 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec112184-f961-42e1-9a9a-129abd1d7341 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:34:03.794237 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec112184-f961-42e1-9a9a-129abd1d7341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:34:03.794510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec112184-f961-42e1-9a9a-129abd1d7341 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:34:03.795246 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec112184-f961-42e1-9a9a-129abd1d7341 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:34:03.796844 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec112184-f961-42e1-9a9a-129abd1d7341 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:34:03.808944 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1200/2398] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:34:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 10:34:03.813232 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1c02d-2be2-48f8-969b-c0625a4e4cbf admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:34:03.823513 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1c02d-2be2-48f8-969b-c0625a4e4cbf admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:34:03.828548 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1525753-7ca7-419d-877a-5a9afffbfbe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:34:03.829222 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1525753-7ca7-419d-877a-5a9afffbfbe9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:34:03.829502 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1525753-7ca7-419d-877a-5a9afffbfbe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:34:03.829804 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1525753-7ca7-419d-877a-5a9afffbfbe9 admin admin] PATH_INFO: `/v3/projects/2bddd4919bbd4fcdbb4fba779e17f9fe/users/285b018c9220408f8f6aefd9ed6f5c51/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:34:03.844344 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1525753-7ca7-419d-877a-5a9afffbfbe9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bddd4919bbd4fcdbb4fba779e17f9fe, user_id=285b018c9220408f8f6aefd9ed6f5c51, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:34:03.847157 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1525753-7ca7-419d-877a-5a9afffbfbe9 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:34:03.862559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1199/2399] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:34:03 2022] PUT /identity/v3/projects/2bddd4919bbd4fcdbb4fba779e17f9fe/users/285b018c9220408f8f6aefd9ed6f5c51/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:34:03.873374 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1bf1a2-37cc-4356-9a64-d2ff103fa918 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:34:03.873664 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1bf1a2-37cc-4356-9a64-d2ff103fa918 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:34:03.873938 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1bf1a2-37cc-4356-9a64-d2ff103fa918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:34:03.905301 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-be1bf1a2-37cc-4356-9a64-d2ff103fa918 None None] MFA Rules not processed for user `285b018c9220408f8f6aefd9ed6f5c51`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:34:03.906531 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-be1bf1a2-37cc-4356-9a64-d2ff103fa918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:34:03.976580 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1201/2400] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:34:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:34:03.988593 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1525753-7ca7-419d-877a-5a9afffbfbe9 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:34:03.990450 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c1525753-7ca7-419d-877a-5a9afffbfbe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:34:04.025115 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:34:04.055946 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1525753-7ca7-419d-877a-5a9afffbfbe9 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:34:04.060253 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05aa52c-05fa-4515-8908-97f7becb35f2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac0a8cf7f8b74e88a04444ec38547528', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:34:04.060873 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05aa52c-05fa-4515-8908-97f7becb35f2 service glance] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:34:04.061164 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05aa52c-05fa-4515-8908-97f7becb35f2 service glance] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:34:04.061433 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05aa52c-05fa-4515-8908-97f7becb35f2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:34:04.071422 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05aa52c-05fa-4515-8908-97f7becb35f2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:34:04.072861 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05aa52c-05fa-4515-8908-97f7becb35f2 service glance] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:34:04.082138 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1200/2401] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:34:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:34:11.845899 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c9b24-dccb-4a34-817c-eab59d709ade None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:34:11.846452 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c9b24-dccb-4a34-817c-eab59d709ade None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:34:11.846452 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c9b24-dccb-4a34-817c-eab59d709ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:34:11.879628 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-022c9b24-dccb-4a34-817c-eab59d709ade None None] MFA Rules not processed for user `706dbfa719c04fd0a3704ebd7eb812cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:34:11.880856 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-022c9b24-dccb-4a34-817c-eab59d709ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:34:11.900891 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:34:11.958314 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1202/2402] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:34:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:34:14.078209 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=14 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:34:21.956433 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=22 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:34:50.477946 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05aa52c-05fa-4515-8908-97f7becb35f2 service glance] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:34:50.593519 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05aa52c-05fa-4515-8908-97f7becb35f2 service glance] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:34:50.598674 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4457ed-6060-443e-a734-26bd3670805c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:34:50.599343 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4457ed-6060-443e-a734-26bd3670805c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:34:50.599625 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4457ed-6060-443e-a734-26bd3670805c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:34:50.599895 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4457ed-6060-443e-a734-26bd3670805c admin admin] PATH_INFO: `/v3/users/285b018c9220408f8f6aefd9ed6f5c51` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:34:50.601131 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4457ed-6060-443e-a734-26bd3670805c admin admin] RBAC: Authorizing `identity:delete_user(user_id=285b018c9220408f8f6aefd9ed6f5c51)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:34:50.603323 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4457ed-6060-443e-a734-26bd3670805c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:34:50.659367 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-cd4457ed-6060-443e-a734-26bd3670805c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '285b018c9220408f8f6aefd9ed6f5c51'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:34:50.667917 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-cd4457ed-6060-443e-a734-26bd3670805c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '285b018c9220408f8f6aefd9ed6f5c51'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:34:50.668560 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-cd4457ed-6060-443e-a734-26bd3670805c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '285b018c9220408f8f6aefd9ed6f5c51'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:34:50.680274 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-cd4457ed-6060-443e-a734-26bd3670805c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '285b018c9220408f8f6aefd9ed6f5c51'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:34:50.680864 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-cd4457ed-6060-443e-a734-26bd3670805c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '285b018c9220408f8f6aefd9ed6f5c51'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:34:50.692667 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1201/2403] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:34:50 2022] DELETE /identity/v3/users/285b018c9220408f8f6aefd9ed6f5c51 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:34:50.697813 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d013dc-cc82-42a6-b5c0-b7092b7c622a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:34:50.698313 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d013dc-cc82-42a6-b5c0-b7092b7c622a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:34:50.698313 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d013dc-cc82-42a6-b5c0-b7092b7c622a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:34:50.739699 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-71d013dc-cc82-42a6-b5c0-b7092b7c622a None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:34:50.740898 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-71d013dc-cc82-42a6-b5c0-b7092b7c622a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:34:50.818109 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1203/2404] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:34:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:34:50.828006 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4457ed-6060-443e-a734-26bd3670805c admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:34:50.829523 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-cd4457ed-6060-443e-a734-26bd3670805c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:34:50.874280 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:34:50.933101 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4457ed-6060-443e-a734-26bd3670805c admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:34:50.938466 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0595be-0f75-4188-9c73-0903bbd30197 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:34:50.939209 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0595be-0f75-4188-9c73-0903bbd30197 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:34:50.939555 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0595be-0f75-4188-9c73-0903bbd30197 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:34:50.939895 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0595be-0f75-4188-9c73-0903bbd30197 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:34:50.950772 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0595be-0f75-4188-9c73-0903bbd30197 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:34:50.952320 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0595be-0f75-4188-9c73-0903bbd30197 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:34:50.963637 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1202/2405] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:34:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:34:51.132853 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d77946-5576-4bb9-9bed-55f0b777ca3f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:34:51.133176 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d77946-5576-4bb9-9bed-55f0b777ca3f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:34:51.134025 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d77946-5576-4bb9-9bed-55f0b777ca3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:34:51.169677 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e9d77946-5576-4bb9-9bed-55f0b777ca3f None None] MFA Rules not processed for user `52fc3ba90dc045c2a5b7bb730da96acc`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:34:51.170915 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e9d77946-5576-4bb9-9bed-55f0b777ca3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:34:51.243910 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1204/2406] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 5 10:34:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:34:51.294878 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0595be-0f75-4188-9c73-0903bbd30197 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:34:51.296573 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-bf0595be-0f75-4188-9c73-0903bbd30197 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:34:51.352784 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d77946-5576-4bb9-9bed-55f0b777ca3f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:34:51.354500 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e9d77946-5576-4bb9-9bed-55f0b777ca3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:34:51.376869 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d77946-5576-4bb9-9bed-55f0b777ca3f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:34:51.383181 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a3179f-b109-49c4-a15a-9af4c43bae74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:34:51.383996 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3179f-b109-49c4-a15a-9af4c43bae74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:34:51.384305 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3179f-b109-49c4-a15a-9af4c43bae74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:34:51.385036 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3179f-b109-49c4-a15a-9af4c43bae74 admin admin] PATH_INFO: `/v3/projects/2bddd4919bbd4fcdbb4fba779e17f9fe` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:34:51.386065 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a3179f-b109-49c4-a15a-9af4c43bae74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bddd4919bbd4fcdbb4fba779e17f9fe)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:34:51.388072 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a3179f-b109-49c4-a15a-9af4c43bae74 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:34:51.402514 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0595be-0f75-4188-9c73-0903bbd30197 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:34:51.407661 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a993e713-6c0d-4b80-a4dc-7ff81d7d422b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c80cf0e429f04f229aea1aa9293a93b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:34:51.408283 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993e713-6c0d-4b80-a4dc-7ff81d7d422b service swift] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:34:51.408562 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993e713-6c0d-4b80-a4dc-7ff81d7d422b service swift] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:34:51.408830 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993e713-6c0d-4b80-a4dc-7ff81d7d422b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:34:51.417709 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a993e713-6c0d-4b80-a4dc-7ff81d7d422b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:34:51.419353 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a993e713-6c0d-4b80-a4dc-7ff81d7d422b service swift] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:34:51.427512 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1203/2407] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 5 10:34:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 10:34:51.441148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c3a3179f-b109-49c4-a15a-9af4c43bae74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bddd4919bbd4fcdbb4fba779e17f9fe', 'request_id': 'req-c3a3179f-b109-49c4-a15a-9af4c43bae74'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:34:51.452611 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c3a3179f-b109-49c4-a15a-9af4c43bae74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bddd4919bbd4fcdbb4fba779e17f9fe', 'request_id': 'req-c3a3179f-b109-49c4-a15a-9af4c43bae74'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:34:51.460408 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c3a3179f-b109-49c4-a15a-9af4c43bae74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bddd4919bbd4fcdbb4fba779e17f9fe', 'request_id': 'req-c3a3179f-b109-49c4-a15a-9af4c43bae74'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:34:51.462878 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:34:51.475100 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c3a3179f-b109-49c4-a15a-9af4c43bae74 admin admin] The token cache is being invalidate because project 2bddd4919bbd4fcdbb4fba779e17f9fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:34:51.475830 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c3a3179f-b109-49c4-a15a-9af4c43bae74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:34:51.476792 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c3a3179f-b109-49c4-a15a-9af4c43bae74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:34:51.479108 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1205/2408] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:34:51 2022] DELETE /identity/v3/projects/2bddd4919bbd4fcdbb4fba779e17f9fe => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:34:51.484120 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23a90c-8f7e-4685-bf27-c16928ceae51 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:34:51.484640 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23a90c-8f7e-4685-bf27-c16928ceae51 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:34:51.484640 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23a90c-8f7e-4685-bf27-c16928ceae51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:34:51.541333 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-1c23a90c-8f7e-4685-bf27-c16928ceae51 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:34:51.542581 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1c23a90c-8f7e-4685-bf27-c16928ceae51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:34:51.621220 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1204/2409] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:34:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:34:51.630976 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153846a-a85e-4fd7-adc5-fa83b7eda154 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:34:51.631303 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153846a-a85e-4fd7-adc5-fa83b7eda154 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:34:51.631543 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153846a-a85e-4fd7-adc5-fa83b7eda154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:34:51.653839 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1153846a-a85e-4fd7-adc5-fa83b7eda154 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:34:51.655427 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1153846a-a85e-4fd7-adc5-fa83b7eda154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:34:51.663964 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1206/2410] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:34:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:34:51.668673 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23a90c-8f7e-4685-bf27-c16928ceae51 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:34:51.678861 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23a90c-8f7e-4685-bf27-c16928ceae51 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:34:51.684001 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bcdcf2-6da1-45d3-92da-abb813781532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:34:51.685454 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bcdcf2-6da1-45d3-92da-abb813781532 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:34:51.685742 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bcdcf2-6da1-45d3-92da-abb813781532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:34:51.686012 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bcdcf2-6da1-45d3-92da-abb813781532 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:34:51.686723 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bcdcf2-6da1-45d3-92da-abb813781532 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:34:51.687060 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bcdcf2-6da1-45d3-92da-abb813781532 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:34:51.688689 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bcdcf2-6da1-45d3-92da-abb813781532 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:34:51.696730 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:34:51.704803 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1205/2411] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:34:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:34:51.708908 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1153846a-a85e-4fd7-adc5-fa83b7eda154 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:34:51.719400 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1153846a-a85e-4fd7-adc5-fa83b7eda154 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:34:51.724933 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d20c5-784f-4ada-9f06-0859c6ad6172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:34:51.725583 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d20c5-784f-4ada-9f06-0859c6ad6172 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:34:51.725873 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d20c5-784f-4ada-9f06-0859c6ad6172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:34:51.726133 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d20c5-784f-4ada-9f06-0859c6ad6172 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:34:51.726837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3d20c5-784f-4ada-9f06-0859c6ad6172 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:34:51.728847 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3d20c5-784f-4ada-9f06-0859c6ad6172 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:34:51.766159 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1207/2412] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:34:51 2022] POST /identity/v3/projects => generated 363 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:34:51.770785 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bcdcf2-6da1-45d3-92da-abb813781532 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:34:51.784621 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bcdcf2-6da1-45d3-92da-abb813781532 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:34:51.857384 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84813b7c-edd5-4e02-b02d-045eac5058b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:34:51.858018 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84813b7c-edd5-4e02-b02d-045eac5058b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:34:51.858295 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84813b7c-edd5-4e02-b02d-045eac5058b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:34:51.858561 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84813b7c-edd5-4e02-b02d-045eac5058b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:34:51.859288 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84813b7c-edd5-4e02-b02d-045eac5058b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:34:51.861505 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84813b7c-edd5-4e02-b02d-045eac5058b9 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:34:51.890956 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1206/2413] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:34:51 2022] POST /identity/v3/users => generated 282 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:34:51.901069 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d20c5-784f-4ada-9f06-0859c6ad6172 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:34:51.912089 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d20c5-784f-4ada-9f06-0859c6ad6172 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:34:51.917490 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed0100-883f-4eb9-a413-3df59cd382ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:34:51.918191 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed0100-883f-4eb9-a413-3df59cd382ea admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:34:51.918472 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed0100-883f-4eb9-a413-3df59cd382ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:34:51.918740 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed0100-883f-4eb9-a413-3df59cd382ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:34:51.919469 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ed0100-883f-4eb9-a413-3df59cd382ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:34:51.921112 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ed0100-883f-4eb9-a413-3df59cd382ea admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:34:51.932577 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1208/2414] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:34:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:34:51.937739 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84813b7c-edd5-4e02-b02d-045eac5058b9 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:34:51.950110 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84813b7c-edd5-4e02-b02d-045eac5058b9 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:34:51.956690 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aabe6e3-6075-4940-9821-2d185cfa0950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:34:51.957334 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aabe6e3-6075-4940-9821-2d185cfa0950 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:34:51.957621 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aabe6e3-6075-4940-9821-2d185cfa0950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:34:51.957901 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aabe6e3-6075-4940-9821-2d185cfa0950 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:34:51.958607 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aabe6e3-6075-4940-9821-2d185cfa0950 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:34:51.960343 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aabe6e3-6075-4940-9821-2d185cfa0950 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:34:51.973153 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1207/2415] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:34:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:34:51.977562 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed0100-883f-4eb9-a413-3df59cd382ea admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:34:51.989898 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed0100-883f-4eb9-a413-3df59cd382ea admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:34:51.995312 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae711e08-4ba5-49a0-9e59-3b4e0f5dcc2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:34:51.996000 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae711e08-4ba5-49a0-9e59-3b4e0f5dcc2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:34:51.996279 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae711e08-4ba5-49a0-9e59-3b4e0f5dcc2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:34:51.996557 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae711e08-4ba5-49a0-9e59-3b4e0f5dcc2f admin admin] PATH_INFO: `/v3/projects/eaf3db7cfc7042409b9aadfac2d34abf/users/b2f2fae347204fc69776ec57526c5d9c/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:34:52.014210 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae711e08-4ba5-49a0-9e59-3b4e0f5dcc2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaf3db7cfc7042409b9aadfac2d34abf, user_id=b2f2fae347204fc69776ec57526c5d9c, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:34:52.016900 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae711e08-4ba5-49a0-9e59-3b4e0f5dcc2f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:34:52.035629 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1209/2416] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:34:51 2022] PUT /identity/v3/projects/eaf3db7cfc7042409b9aadfac2d34abf/users/b2f2fae347204fc69776ec57526c5d9c/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:34:52.047580 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e6b4b-6fe2-4808-8ba6-9db4e93d7e5c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:34:52.048285 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e6b4b-6fe2-4808-8ba6-9db4e93d7e5c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:34:52.048570 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e6b4b-6fe2-4808-8ba6-9db4e93d7e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:34:52.081639 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c55e6b4b-6fe2-4808-8ba6-9db4e93d7e5c None None] MFA Rules not processed for user `b2f2fae347204fc69776ec57526c5d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:34:52.083617 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c55e6b4b-6fe2-4808-8ba6-9db4e93d7e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:34:52.175523 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1208/2417] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:34:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:34:52.187530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae711e08-4ba5-49a0-9e59-3b4e0f5dcc2f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:34:52.190416 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ae711e08-4ba5-49a0-9e59-3b4e0f5dcc2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:34:52.308759 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae711e08-4ba5-49a0-9e59-3b4e0f5dcc2f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:34:52.313305 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6385e4cc-43d0-4c76-a692-a44642217090 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:34:52.314141 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385e4cc-43d0-4c76-a692-a44642217090 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:34:52.314580 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385e4cc-43d0-4c76-a692-a44642217090 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:34:52.315195 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385e4cc-43d0-4c76-a692-a44642217090 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:34:52.325940 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6385e4cc-43d0-4c76-a692-a44642217090 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:34:52.329058 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6385e4cc-43d0-4c76-a692-a44642217090 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:34:52.343666 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1210/2418] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:34:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:35:02.171947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=67,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:35:02.337360 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=64,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:35:07.399918 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55e6b4b-6fe2-4808-8ba6-9db4e93d7e5c None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:35:07.402779 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c55e6b4b-6fe2-4808-8ba6-9db4e93d7e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:35:07.502362 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55e6b4b-6fe2-4808-8ba6-9db4e93d7e5c None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:35:07.507978 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71120c5b-1aba-46fd-83e2-b40ae874cc25 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8743f5bb0be54d57a957617636b0b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:35:07.508594 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71120c5b-1aba-46fd-83e2-b40ae874cc25 service placement] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:35:07.508871 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71120c5b-1aba-46fd-83e2-b40ae874cc25 service placement] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:35:07.509145 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71120c5b-1aba-46fd-83e2-b40ae874cc25 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:35:07.511632 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-71120c5b-1aba-46fd-83e2-b40ae874cc25 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:35:07.523603 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71120c5b-1aba-46fd-83e2-b40ae874cc25 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:35:07.525200 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71120c5b-1aba-46fd-83e2-b40ae874cc25 service placement] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:35:07.535691 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1209/2419] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:35:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:35:10.994256 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce367069-6d60-4b7f-be39-503d2e0368dd None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:35:10.994734 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce367069-6d60-4b7f-be39-503d2e0368dd None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:35:10.994834 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce367069-6d60-4b7f-be39-503d2e0368dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:35:11.015032 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ce367069-6d60-4b7f-be39-503d2e0368dd None None] MFA Rules not processed for user `52fc3ba90dc045c2a5b7bb730da96acc`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:35:11.016314 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ce367069-6d60-4b7f-be39-503d2e0368dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:35:11.078094 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1211/2420] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 5 10:35:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:35:11.090652 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71120c5b-1aba-46fd-83e2-b40ae874cc25 service placement] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:35:11.092397 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-71120c5b-1aba-46fd-83e2-b40ae874cc25 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:35:11.172068 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71120c5b-1aba-46fd-83e2-b40ae874cc25 service placement] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:35:11.177746 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec56497b-c139-4e7b-b13b-22c0c7405968 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c80cf0e429f04f229aea1aa9293a93b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:35:11.178377 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec56497b-c139-4e7b-b13b-22c0c7405968 service swift] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:35:11.178660 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec56497b-c139-4e7b-b13b-22c0c7405968 service swift] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:35:11.178935 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec56497b-c139-4e7b-b13b-22c0c7405968 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:35:11.188085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec56497b-c139-4e7b-b13b-22c0c7405968 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:35:11.189593 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec56497b-c139-4e7b-b13b-22c0c7405968 service swift] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:35:11.198259 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1210/2421] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 5 10:35:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 10:35:21.070532 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=25 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:35:21.194930 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=73 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:35:38.292598 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce367069-6d60-4b7f-be39-503d2e0368dd None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:35:38.294286 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ce367069-6d60-4b7f-be39-503d2e0368dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:35:38.357122 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce367069-6d60-4b7f-be39-503d2e0368dd None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:35:38.361570 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1f775d-0d12-47d7-ae5b-8eb23c90d8b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:35:38.365028 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f775d-0d12-47d7-ae5b-8eb23c90d8b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:35:38.365028 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f775d-0d12-47d7-ae5b-8eb23c90d8b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:35:38.365028 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f775d-0d12-47d7-ae5b-8eb23c90d8b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:35:38.365028 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ee1f775d-0d12-47d7-ae5b-8eb23c90d8b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:35:38.428327 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1f775d-0d12-47d7-ae5b-8eb23c90d8b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:35:38.429705 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1f775d-0d12-47d7-ae5b-8eb23c90d8b8 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:35:38.438874 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1212/2422] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:35:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:35:39.077410 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb71dae-8e21-4254-86b5-ed306c0d6d64 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:35:39.077916 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb71dae-8e21-4254-86b5-ed306c0d6d64 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:35:39.078021 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb71dae-8e21-4254-86b5-ed306c0d6d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:35:39.100362 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-1fb71dae-8e21-4254-86b5-ed306c0d6d64 None None] MFA Rules not processed for user `52fc3ba90dc045c2a5b7bb730da96acc`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:35:39.101619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1fb71dae-8e21-4254-86b5-ed306c0d6d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:35:39.110780 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1211/2423] 10.222.0.40 () {58 vars in 967 bytes} [Fri Aug 5 10:35:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:35:39.124124 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1f775d-0d12-47d7-ae5b-8eb23c90d8b8 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:35:39.136678 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1f775d-0d12-47d7-ae5b-8eb23c90d8b8 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:35:39.141713 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd4dc72-cc25-498b-be4e-69de65149bcd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c80cf0e429f04f229aea1aa9293a93b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:35:39.142702 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4dc72-cc25-498b-be4e-69de65149bcd service swift] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:35:39.143014 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4dc72-cc25-498b-be4e-69de65149bcd service swift] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:35:39.143283 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4dc72-cc25-498b-be4e-69de65149bcd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:35:39.152396 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd4dc72-cc25-498b-be4e-69de65149bcd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:35:39.153998 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd4dc72-cc25-498b-be4e-69de65149bcd service swift] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:35:39.163371 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1213/2424] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 5 10:35:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 10:35:40.927667 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb71dae-8e21-4254-86b5-ed306c0d6d64 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:35:40.992836 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb71dae-8e21-4254-86b5-ed306c0d6d64 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:35:40.997792 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dca057-ce9c-43ac-b8c3-bf8f7154e2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:35:40.998448 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dca057-ce9c-43ac-b8c3-bf8f7154e2a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:35:40.998736 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dca057-ce9c-43ac-b8c3-bf8f7154e2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:35:40.999052 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dca057-ce9c-43ac-b8c3-bf8f7154e2a9 admin admin] PATH_INFO: `/v3/users/b2f2fae347204fc69776ec57526c5d9c` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:35:41.000240 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dca057-ce9c-43ac-b8c3-bf8f7154e2a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2f2fae347204fc69776ec57526c5d9c)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:35:41.002191 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dca057-ce9c-43ac-b8c3-bf8f7154e2a9 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:35:41.052985 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-44dca057-ce9c-43ac-b8c3-bf8f7154e2a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2f2fae347204fc69776ec57526c5d9c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:35:41.062342 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-44dca057-ce9c-43ac-b8c3-bf8f7154e2a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2f2fae347204fc69776ec57526c5d9c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:35:41.063019 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-44dca057-ce9c-43ac-b8c3-bf8f7154e2a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2f2fae347204fc69776ec57526c5d9c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:35:41.074855 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-44dca057-ce9c-43ac-b8c3-bf8f7154e2a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2f2fae347204fc69776ec57526c5d9c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:35:41.075496 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-44dca057-ce9c-43ac-b8c3-bf8f7154e2a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2f2fae347204fc69776ec57526c5d9c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:35:41.088116 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1212/2425] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:35:40 2022] DELETE /identity/v3/users/b2f2fae347204fc69776ec57526c5d9c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:35:41.093017 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d8b5e0-7bed-4791-8275-839b6f36ee12 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:35:41.093526 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d8b5e0-7bed-4791-8275-839b6f36ee12 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:35:41.093635 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d8b5e0-7bed-4791-8275-839b6f36ee12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:35:41.115770 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-40d8b5e0-7bed-4791-8275-839b6f36ee12 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:35:41.117052 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-40d8b5e0-7bed-4791-8275-839b6f36ee12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:35:41.178742 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1214/2426] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:35:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:35:41.189024 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dca057-ce9c-43ac-b8c3-bf8f7154e2a9 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:35:41.190763 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-44dca057-ce9c-43ac-b8c3-bf8f7154e2a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:35:41.256044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dca057-ce9c-43ac-b8c3-bf8f7154e2a9 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:35:41.259707 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162a056-4793-41b4-a42e-c43981e728cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:35:41.260331 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162a056-4793-41b4-a42e-c43981e728cd service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:35:41.260623 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162a056-4793-41b4-a42e-c43981e728cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:35:41.260873 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162a056-4793-41b4-a42e-c43981e728cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:35:41.271059 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8162a056-4793-41b4-a42e-c43981e728cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:35:41.272506 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8162a056-4793-41b4-a42e-c43981e728cd service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:35:41.282047 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1213/2427] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:35:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:35:41.639059 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d8b5e0-7bed-4791-8275-839b6f36ee12 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:35:41.640645 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-40d8b5e0-7bed-4791-8275-839b6f36ee12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:35:41.660787 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d8b5e0-7bed-4791-8275-839b6f36ee12 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:35:41.668358 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcba2e7-4954-4567-bb9f-ca5075b1b920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:35:41.668985 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcba2e7-4954-4567-bb9f-ca5075b1b920 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:35:41.669419 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcba2e7-4954-4567-bb9f-ca5075b1b920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:35:41.669604 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcba2e7-4954-4567-bb9f-ca5075b1b920 admin admin] PATH_INFO: `/v3/projects/eaf3db7cfc7042409b9aadfac2d34abf` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:35:41.670841 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcba2e7-4954-4567-bb9f-ca5075b1b920 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaf3db7cfc7042409b9aadfac2d34abf)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:35:41.673007 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcba2e7-4954-4567-bb9f-ca5075b1b920 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:35:41.716984 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:35:41.727041 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4bcba2e7-4954-4567-bb9f-ca5075b1b920 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaf3db7cfc7042409b9aadfac2d34abf', 'request_id': 'req-4bcba2e7-4954-4567-bb9f-ca5075b1b920'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:35:41.736078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4bcba2e7-4954-4567-bb9f-ca5075b1b920 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaf3db7cfc7042409b9aadfac2d34abf', 'request_id': 'req-4bcba2e7-4954-4567-bb9f-ca5075b1b920'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:35:41.742865 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4bcba2e7-4954-4567-bb9f-ca5075b1b920 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaf3db7cfc7042409b9aadfac2d34abf', 'request_id': 'req-4bcba2e7-4954-4567-bb9f-ca5075b1b920'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:35:41.755184 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4bcba2e7-4954-4567-bb9f-ca5075b1b920 admin admin] The token cache is being invalidate because project eaf3db7cfc7042409b9aadfac2d34abf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:35:41.755489 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4bcba2e7-4954-4567-bb9f-ca5075b1b920 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:35:41.756193 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4bcba2e7-4954-4567-bb9f-ca5075b1b920 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:35:41.758219 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1215/2428] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:35:41 2022] DELETE /identity/v3/projects/eaf3db7cfc7042409b9aadfac2d34abf => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:35:44.470229 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162a056-4793-41b4-a42e-c43981e728cd service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:35:44.479576 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162a056-4793-41b4-a42e-c43981e728cd service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:35:44.491633 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:35:44.532766 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a28adf-4c98-4d82-a7e2-3f2ca2d08202 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:35:44.533394 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a28adf-4c98-4d82-a7e2-3f2ca2d08202 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:35:44.533677 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a28adf-4c98-4d82-a7e2-3f2ca2d08202 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:35:44.533949 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a28adf-4c98-4d82-a7e2-3f2ca2d08202 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:35:44.536051 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-63a28adf-4c98-4d82-a7e2-3f2ca2d08202 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:35:44.601525 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a28adf-4c98-4d82-a7e2-3f2ca2d08202 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:35:44.603081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a28adf-4c98-4d82-a7e2-3f2ca2d08202 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:35:44.613128 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1214/2429] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:35:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:35:50.795325 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcba2e7-4954-4567-bb9f-ca5075b1b920 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:35:50.796861 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4bcba2e7-4954-4567-bb9f-ca5075b1b920 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:35:50.813367 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcba2e7-4954-4567-bb9f-ca5075b1b920 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:35:50.818825 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8722a20-c5ed-4c8c-9d21-4d665d47c8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:35:50.819511 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8722a20-c5ed-4c8c-9d21-4d665d47c8d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:35:50.819793 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8722a20-c5ed-4c8c-9d21-4d665d47c8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:35:50.820067 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8722a20-c5ed-4c8c-9d21-4d665d47c8d2 admin admin] PATH_INFO: `/v3/users/a4b9bf1486c745f0bd204307b942809a` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:35:50.821260 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8722a20-c5ed-4c8c-9d21-4d665d47c8d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4b9bf1486c745f0bd204307b942809a)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:35:50.823253 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8722a20-c5ed-4c8c-9d21-4d665d47c8d2 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:35:50.879017 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f8722a20-c5ed-4c8c-9d21-4d665d47c8d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4b9bf1486c745f0bd204307b942809a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:35:50.879897 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f8722a20-c5ed-4c8c-9d21-4d665d47c8d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4b9bf1486c745f0bd204307b942809a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:35:50.880370 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f8722a20-c5ed-4c8c-9d21-4d665d47c8d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4b9bf1486c745f0bd204307b942809a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:35:50.888968 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f8722a20-c5ed-4c8c-9d21-4d665d47c8d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4b9bf1486c745f0bd204307b942809a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:35:50.896847 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f8722a20-c5ed-4c8c-9d21-4d665d47c8d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4b9bf1486c745f0bd204307b942809a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:35:50.913979 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1216/2430] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:35:50 2022] DELETE /identity/v3/users/a4b9bf1486c745f0bd204307b942809a => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:35:51.096600 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a28adf-4c98-4d82-a7e2-3f2ca2d08202 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:35:51.107656 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a28adf-4c98-4d82-a7e2-3f2ca2d08202 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:35:51.181189 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed4eb24-3938-4243-8bdc-bb14fe07a21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:35:51.181772 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4eb24-3938-4243-8bdc-bb14fe07a21e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:35:51.182055 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4eb24-3938-4243-8bdc-bb14fe07a21e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:35:51.182330 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4eb24-3938-4243-8bdc-bb14fe07a21e admin admin] PATH_INFO: `/v3/projects/4183067b2b9c429d85238263419ec3f7` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:35:51.183506 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed4eb24-3938-4243-8bdc-bb14fe07a21e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4183067b2b9c429d85238263419ec3f7)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:35:51.185426 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed4eb24-3938-4243-8bdc-bb14fe07a21e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:35:51.234677 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-aed4eb24-3938-4243-8bdc-bb14fe07a21e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4183067b2b9c429d85238263419ec3f7', 'request_id': 'req-aed4eb24-3938-4243-8bdc-bb14fe07a21e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:35:51.242669 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-aed4eb24-3938-4243-8bdc-bb14fe07a21e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4183067b2b9c429d85238263419ec3f7', 'request_id': 'req-aed4eb24-3938-4243-8bdc-bb14fe07a21e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:35:51.254958 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-aed4eb24-3938-4243-8bdc-bb14fe07a21e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4183067b2b9c429d85238263419ec3f7', 'request_id': 'req-aed4eb24-3938-4243-8bdc-bb14fe07a21e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:35:51.260676 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:35:51.283483 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-aed4eb24-3938-4243-8bdc-bb14fe07a21e admin admin] The token cache is being invalidate because project 4183067b2b9c429d85238263419ec3f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:35:51.284326 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-aed4eb24-3938-4243-8bdc-bb14fe07a21e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:35:51.285321 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-aed4eb24-3938-4243-8bdc-bb14fe07a21e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:35:51.294075 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8722a20-c5ed-4c8c-9d21-4d665d47c8d2 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:35:51.294874 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1215/2431] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:35:51 2022] DELETE /identity/v3/projects/4183067b2b9c429d85238263419ec3f7 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:35:51.296023 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f8722a20-c5ed-4c8c-9d21-4d665d47c8d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:35:51.310632 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8722a20-c5ed-4c8c-9d21-4d665d47c8d2 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:35:51.376606 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb332947-a1bc-45b8-bea8-2dff0de5d3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:35:51.377262 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb332947-a1bc-45b8-bea8-2dff0de5d3d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:35:51.377537 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb332947-a1bc-45b8-bea8-2dff0de5d3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:35:51.377804 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb332947-a1bc-45b8-bea8-2dff0de5d3d4 admin admin] PATH_INFO: `/v3/users/d155329c404d4e1ba27094e57210b8d7` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:35:51.379008 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb332947-a1bc-45b8-bea8-2dff0de5d3d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d155329c404d4e1ba27094e57210b8d7)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:35:51.381316 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb332947-a1bc-45b8-bea8-2dff0de5d3d4 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:35:51.429330 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:35:51.435909 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cb332947-a1bc-45b8-bea8-2dff0de5d3d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd155329c404d4e1ba27094e57210b8d7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:35:51.436765 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cb332947-a1bc-45b8-bea8-2dff0de5d3d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd155329c404d4e1ba27094e57210b8d7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:35:51.437770 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cb332947-a1bc-45b8-bea8-2dff0de5d3d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd155329c404d4e1ba27094e57210b8d7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:35:51.446569 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cb332947-a1bc-45b8-bea8-2dff0de5d3d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd155329c404d4e1ba27094e57210b8d7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:35:51.454270 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cb332947-a1bc-45b8-bea8-2dff0de5d3d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd155329c404d4e1ba27094e57210b8d7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:35:51.471830 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1217/2432] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:35:51 2022] DELETE /identity/v3/users/d155329c404d4e1ba27094e57210b8d7 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:35:51.660831 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed4eb24-3938-4243-8bdc-bb14fe07a21e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:35:51.662407 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-aed4eb24-3938-4243-8bdc-bb14fe07a21e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:35:51.675398 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed4eb24-3938-4243-8bdc-bb14fe07a21e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:35:51.734111 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c8c0e-6173-4ba5-bb27-50a8c9aa4ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:35:51.734755 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c8c0e-6173-4ba5-bb27-50a8c9aa4ded admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:35:51.735047 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c8c0e-6173-4ba5-bb27-50a8c9aa4ded admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:35:51.735336 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c8c0e-6173-4ba5-bb27-50a8c9aa4ded admin admin] PATH_INFO: `/v3/projects/bfc491299375406ab19f0723bef93f14` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:35:51.736513 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368c8c0e-6173-4ba5-bb27-50a8c9aa4ded admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfc491299375406ab19f0723bef93f14)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:35:51.738434 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368c8c0e-6173-4ba5-bb27-50a8c9aa4ded admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:35:51.799177 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-368c8c0e-6173-4ba5-bb27-50a8c9aa4ded admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfc491299375406ab19f0723bef93f14', 'request_id': 'req-368c8c0e-6173-4ba5-bb27-50a8c9aa4ded'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:35:51.807454 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-368c8c0e-6173-4ba5-bb27-50a8c9aa4ded admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfc491299375406ab19f0723bef93f14', 'request_id': 'req-368c8c0e-6173-4ba5-bb27-50a8c9aa4ded'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:35:51.817839 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-368c8c0e-6173-4ba5-bb27-50a8c9aa4ded admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfc491299375406ab19f0723bef93f14', 'request_id': 'req-368c8c0e-6173-4ba5-bb27-50a8c9aa4ded'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:35:51.829221 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-368c8c0e-6173-4ba5-bb27-50a8c9aa4ded admin admin] The token cache is being invalidate because project bfc491299375406ab19f0723bef93f14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:35:51.829503 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-368c8c0e-6173-4ba5-bb27-50a8c9aa4ded admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:35:51.830170 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-368c8c0e-6173-4ba5-bb27-50a8c9aa4ded admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:35:51.831756 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1216/2433] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:35:51 2022] DELETE /identity/v3/projects/bfc491299375406ab19f0723bef93f14 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:01.468195 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:DELETE=5,keystone:SELECT=14,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:01.826154 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=53,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:05.996939 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb332947-a1bc-45b8-bea8-2dff0de5d3d4 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:05.998488 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cb332947-a1bc-45b8-bea8-2dff0de5d3d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:06.056451 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb332947-a1bc-45b8-bea8-2dff0de5d3d4 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:06.106855 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa410b9d-5f2a-4694-ad5d-ee7e29143943 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:06.107807 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa410b9d-5f2a-4694-ad5d-ee7e29143943 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:06.108190 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa410b9d-5f2a-4694-ad5d-ee7e29143943 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:06.108530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa410b9d-5f2a-4694-ad5d-ee7e29143943 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:06.111462 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-aa410b9d-5f2a-4694-ad5d-ee7e29143943 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:06.184828 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa410b9d-5f2a-4694-ad5d-ee7e29143943 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:06.186279 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa410b9d-5f2a-4694-ad5d-ee7e29143943 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:06.196344 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1218/2434] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:36:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:36:09.479126 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c8c0e-6173-4ba5-bb27-50a8c9aa4ded admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:09.481291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-368c8c0e-6173-4ba5-bb27-50a8c9aa4ded admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:09.495758 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c8c0e-6173-4ba5-bb27-50a8c9aa4ded admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:09.501079 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27700a1-dce0-4c3b-ad64-81e88b9618a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:09.501726 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27700a1-dce0-4c3b-ad64-81e88b9618a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:09.502005 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27700a1-dce0-4c3b-ad64-81e88b9618a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:09.502274 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27700a1-dce0-4c3b-ad64-81e88b9618a0 admin admin] PATH_INFO: `/v3/users/7e7eb355fac04252a4be1f8e60037924` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:09.517516 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27700a1-dce0-4c3b-ad64-81e88b9618a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e7eb355fac04252a4be1f8e60037924)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:09.519496 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27700a1-dce0-4c3b-ad64-81e88b9618a0 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:09.575237 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d27700a1-dce0-4c3b-ad64-81e88b9618a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e7eb355fac04252a4be1f8e60037924'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:09.584710 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d27700a1-dce0-4c3b-ad64-81e88b9618a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e7eb355fac04252a4be1f8e60037924'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:09.585541 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d27700a1-dce0-4c3b-ad64-81e88b9618a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e7eb355fac04252a4be1f8e60037924'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:09.597795 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d27700a1-dce0-4c3b-ad64-81e88b9618a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e7eb355fac04252a4be1f8e60037924'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:09.598503 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d27700a1-dce0-4c3b-ad64-81e88b9618a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e7eb355fac04252a4be1f8e60037924'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:09.616144 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1217/2435] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:36:09 2022] DELETE /identity/v3/users/7e7eb355fac04252a4be1f8e60037924 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:09.815990 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa410b9d-5f2a-4694-ad5d-ee7e29143943 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:09.831480 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa410b9d-5f2a-4694-ad5d-ee7e29143943 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:09.897922 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eda8bc-9bc4-46c0-b960-b8618805ea4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:09.898570 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eda8bc-9bc4-46c0-b960-b8618805ea4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:09.898847 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eda8bc-9bc4-46c0-b960-b8618805ea4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:09.899152 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eda8bc-9bc4-46c0-b960-b8618805ea4f admin admin] PATH_INFO: `/v3/projects/40807f4af05048deae68cf3515433261` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:09.908023 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:09.910952 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eda8bc-9bc4-46c0-b960-b8618805ea4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=40807f4af05048deae68cf3515433261)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:09.913541 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eda8bc-9bc4-46c0-b960-b8618805ea4f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:09.962346 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-96eda8bc-9bc4-46c0-b960-b8618805ea4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40807f4af05048deae68cf3515433261', 'request_id': 'req-96eda8bc-9bc4-46c0-b960-b8618805ea4f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:09.972086 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-96eda8bc-9bc4-46c0-b960-b8618805ea4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40807f4af05048deae68cf3515433261', 'request_id': 'req-96eda8bc-9bc4-46c0-b960-b8618805ea4f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:09.977851 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-96eda8bc-9bc4-46c0-b960-b8618805ea4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40807f4af05048deae68cf3515433261', 'request_id': 'req-96eda8bc-9bc4-46c0-b960-b8618805ea4f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:09.989451 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-96eda8bc-9bc4-46c0-b960-b8618805ea4f admin admin] The token cache is being invalidate because project 40807f4af05048deae68cf3515433261 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:36:09.989762 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-96eda8bc-9bc4-46c0-b960-b8618805ea4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:09.990374 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-96eda8bc-9bc4-46c0-b960-b8618805ea4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:09.991985 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1219/2436] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:36:09 2022] DELETE /identity/v3/projects/40807f4af05048deae68cf3515433261 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:09.997107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf7da2-7f0e-4e59-b4eb-f942131098c8 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:09.997522 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf7da2-7f0e-4e59-b4eb-f942131098c8 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:09.998025 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf7da2-7f0e-4e59-b4eb-f942131098c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:10.018683 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-16bf7da2-7f0e-4e59-b4eb-f942131098c8 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:10.019937 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-16bf7da2-7f0e-4e59-b4eb-f942131098c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:10.081702 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1218/2437] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:10.089868 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5a259-cdb7-4c67-ba31-ca6ed1ef11e0 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:10.090152 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5a259-cdb7-4c67-ba31-ca6ed1ef11e0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:10.090426 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5a259-cdb7-4c67-ba31-ca6ed1ef11e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:10.110443 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-95e5a259-cdb7-4c67-ba31-ca6ed1ef11e0 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:10.111688 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-95e5a259-cdb7-4c67-ba31-ca6ed1ef11e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:10.120356 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1220/2438] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:10.124553 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bf7da2-7f0e-4e59-b4eb-f942131098c8 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:10.134641 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bf7da2-7f0e-4e59-b4eb-f942131098c8 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:10.140223 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d7803e-9dbf-4bce-820d-47d3f9d2ea70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:10.141166 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7803e-9dbf-4bce-820d-47d3f9d2ea70 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:10.141448 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7803e-9dbf-4bce-820d-47d3f9d2ea70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:10.141716 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7803e-9dbf-4bce-820d-47d3f9d2ea70 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:10.145939 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d7803e-9dbf-4bce-820d-47d3f9d2ea70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:36:10.146252 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d7803e-9dbf-4bce-820d-47d3f9d2ea70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:10.148738 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d7803e-9dbf-4bce-820d-47d3f9d2ea70 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:10.162604 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1219/2439] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:36:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:36:10.167007 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e5a259-cdb7-4c67-ba31-ca6ed1ef11e0 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:10.176887 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e5a259-cdb7-4c67-ba31-ca6ed1ef11e0 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:10.182203 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc66c8-39f1-418a-9b87-0c870bf64e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:10.182836 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc66c8-39f1-418a-9b87-0c870bf64e0e admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:10.183195 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc66c8-39f1-418a-9b87-0c870bf64e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:10.183455 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc66c8-39f1-418a-9b87-0c870bf64e0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:10.184161 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bc66c8-39f1-418a-9b87-0c870bf64e0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:10.186234 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bc66c8-39f1-418a-9b87-0c870bf64e0e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:10.228492 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1221/2440] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:36:10 2022] POST /identity/v3/projects => generated 385 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:36:10.233605 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d7803e-9dbf-4bce-820d-47d3f9d2ea70 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:10.244967 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d7803e-9dbf-4bce-820d-47d3f9d2ea70 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:10.271827 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc66c8-39f1-418a-9b87-0c870bf64e0e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:10.273869 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-68bc66c8-39f1-418a-9b87-0c870bf64e0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:10.296734 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:10.311635 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c7e03e-7974-43ea-b6b1-e46b8f2896b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:10.312311 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7e03e-7974-43ea-b6b1-e46b8f2896b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:10.312591 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7e03e-7974-43ea-b6b1-e46b8f2896b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:10.312863 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7e03e-7974-43ea-b6b1-e46b8f2896b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:10.313574 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c7e03e-7974-43ea-b6b1-e46b8f2896b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:10.315545 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c7e03e-7974-43ea-b6b1-e46b8f2896b5 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:10.334062 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc66c8-39f1-418a-9b87-0c870bf64e0e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:10.337956 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be965a2-1c31-4e59-a95e-9b0ca8477697 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:10.338332 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1220/2441] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:36:10 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:36:10.338577 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be965a2-1c31-4e59-a95e-9b0ca8477697 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:10.338885 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be965a2-1c31-4e59-a95e-9b0ca8477697 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:10.339175 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be965a2-1c31-4e59-a95e-9b0ca8477697 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:10.341035 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3be965a2-1c31-4e59-a95e-9b0ca8477697 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:10.342634 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c7e03e-7974-43ea-b6b1-e46b8f2896b5 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:10.352500 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c7e03e-7974-43ea-b6b1-e46b8f2896b5 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:10.357199 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47649c33-4ddb-431b-8e43-f6803548e5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:10.357836 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47649c33-4ddb-431b-8e43-f6803548e5be admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:10.358110 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47649c33-4ddb-431b-8e43-f6803548e5be admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:10.358390 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47649c33-4ddb-431b-8e43-f6803548e5be admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:10.359128 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47649c33-4ddb-431b-8e43-f6803548e5be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:10.360708 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47649c33-4ddb-431b-8e43-f6803548e5be admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:10.372099 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1221/2442] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 10:36:10.376085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47649c33-4ddb-431b-8e43-f6803548e5be admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:10.385814 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47649c33-4ddb-431b-8e43-f6803548e5be admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:10.391014 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97a573b-85c5-461c-8e1e-d5c3cccd88df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:10.391637 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a573b-85c5-461c-8e1e-d5c3cccd88df admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:10.391913 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a573b-85c5-461c-8e1e-d5c3cccd88df admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:10.392176 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a573b-85c5-461c-8e1e-d5c3cccd88df admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:10.392857 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97a573b-85c5-461c-8e1e-d5c3cccd88df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:10.394412 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97a573b-85c5-461c-8e1e-d5c3cccd88df admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:10.404850 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1222/2443] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 10:36:10.408852 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97a573b-85c5-461c-8e1e-d5c3cccd88df admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:10.409642 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:10.419286 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97a573b-85c5-461c-8e1e-d5c3cccd88df admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:10.424043 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf0b47-8ed2-4d9e-9d73-6648bc424190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:10.424694 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf0b47-8ed2-4d9e-9d73-6648bc424190 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:10.424970 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf0b47-8ed2-4d9e-9d73-6648bc424190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:10.425238 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf0b47-8ed2-4d9e-9d73-6648bc424190 admin admin] PATH_INFO: `/v3/projects/feb60e9ea1de4b5382ee941d4d5f5454/users/fe0bc9f355f947cbbf8b8a638d7a87da/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:10.430944 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be965a2-1c31-4e59-a95e-9b0ca8477697 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:10.432404 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be965a2-1c31-4e59-a95e-9b0ca8477697 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:10.440457 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaf0b47-8ed2-4d9e-9d73-6648bc424190 admin admin] RBAC: Authorizing `identity:create_grant(project_id=feb60e9ea1de4b5382ee941d4d5f5454, user_id=fe0bc9f355f947cbbf8b8a638d7a87da, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:10.443104 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaf0b47-8ed2-4d9e-9d73-6648bc424190 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:10.453397 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1222/2444] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:36:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 10:36:10.457018 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1223/2445] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:36:10 2022] PUT /identity/v3/projects/feb60e9ea1de4b5382ee941d4d5f5454/users/fe0bc9f355f947cbbf8b8a638d7a87da/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:36:10.468891 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c98a7-17c3-49bd-9cc0-530301bc4f3a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:10.469175 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c98a7-17c3-49bd-9cc0-530301bc4f3a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:10.469443 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c98a7-17c3-49bd-9cc0-530301bc4f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:10.499994 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-568c98a7-17c3-49bd-9cc0-530301bc4f3a None None] MFA Rules not processed for user `fe0bc9f355f947cbbf8b8a638d7a87da`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:10.502218 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-568c98a7-17c3-49bd-9cc0-530301bc4f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:10.587588 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1223/2446] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:10.598940 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf0b47-8ed2-4d9e-9d73-6648bc424190 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:10.600808 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8aaf0b47-8ed2-4d9e-9d73-6648bc424190 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:10.679420 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf0b47-8ed2-4d9e-9d73-6648bc424190 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:10.685564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81595a8b-0e8a-4147-8662-fd46d55b1f7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:10.686299 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81595a8b-0e8a-4147-8662-fd46d55b1f7a service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:10.686637 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81595a8b-0e8a-4147-8662-fd46d55b1f7a service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:10.687010 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81595a8b-0e8a-4147-8662-fd46d55b1f7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:10.695626 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81595a8b-0e8a-4147-8662-fd46d55b1f7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:10.697377 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81595a8b-0e8a-4147-8662-fd46d55b1f7a service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:10.711413 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1224/2447] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:36:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:36:10.729834 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c98a7-17c3-49bd-9cc0-530301bc4f3a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:10.797207 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c98a7-17c3-49bd-9cc0-530301bc4f3a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:10.800781 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91161bc7-8ac2-4127-a0df-6a57c5453b6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:10.801408 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91161bc7-8ac2-4127-a0df-6a57c5453b6a service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:10.801687 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91161bc7-8ac2-4127-a0df-6a57c5453b6a service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:10.801957 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91161bc7-8ac2-4127-a0df-6a57c5453b6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:10.803864 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-91161bc7-8ac2-4127-a0df-6a57c5453b6a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:10.817581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91161bc7-8ac2-4127-a0df-6a57c5453b6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:10.819057 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91161bc7-8ac2-4127-a0df-6a57c5453b6a service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:10.830030 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1224/2448] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:36:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:36:19.284541 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81595a8b-0e8a-4147-8662-fd46d55b1f7a service nova] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:19.346059 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81595a8b-0e8a-4147-8662-fd46d55b1f7a service nova] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:19.351386 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b908652-0079-47c6-b3eb-4d71ebc7badd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:19.352028 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b908652-0079-47c6-b3eb-4d71ebc7badd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:19.352314 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b908652-0079-47c6-b3eb-4d71ebc7badd admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:19.352560 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b908652-0079-47c6-b3eb-4d71ebc7badd admin admin] PATH_INFO: `/v3/users/fe0bc9f355f947cbbf8b8a638d7a87da` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:19.353843 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b908652-0079-47c6-b3eb-4d71ebc7badd admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe0bc9f355f947cbbf8b8a638d7a87da)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:19.355957 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b908652-0079-47c6-b3eb-4d71ebc7badd admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:19.378103 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:19.406519 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8b908652-0079-47c6-b3eb-4d71ebc7badd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe0bc9f355f947cbbf8b8a638d7a87da'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:19.415167 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8b908652-0079-47c6-b3eb-4d71ebc7badd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe0bc9f355f947cbbf8b8a638d7a87da'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:19.415785 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8b908652-0079-47c6-b3eb-4d71ebc7badd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe0bc9f355f947cbbf8b8a638d7a87da'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:19.427435 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8b908652-0079-47c6-b3eb-4d71ebc7badd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe0bc9f355f947cbbf8b8a638d7a87da'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:19.428089 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8b908652-0079-47c6-b3eb-4d71ebc7badd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe0bc9f355f947cbbf8b8a638d7a87da'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:19.440267 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1225/2449] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:36:19 2022] DELETE /identity/v3/users/fe0bc9f355f947cbbf8b8a638d7a87da => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:19.444820 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149a9d0-cb27-4f6c-9df0-e49bdf840ac4 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:19.445313 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149a9d0-cb27-4f6c-9df0-e49bdf840ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:19.445439 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149a9d0-cb27-4f6c-9df0-e49bdf840ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:19.467618 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8149a9d0-cb27-4f6c-9df0-e49bdf840ac4 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:19.468836 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8149a9d0-cb27-4f6c-9df0-e49bdf840ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:19.534542 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1225/2450] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:19.546079 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b908652-0079-47c6-b3eb-4d71ebc7badd admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:19.548380 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8b908652-0079-47c6-b3eb-4d71ebc7badd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:19.609148 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b908652-0079-47c6-b3eb-4d71ebc7badd admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:19.612885 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f55f65b-38d0-4ab1-91b7-7b64c7fe8170 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:19.613502 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f55f65b-38d0-4ab1-91b7-7b64c7fe8170 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:19.613784 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f55f65b-38d0-4ab1-91b7-7b64c7fe8170 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:19.614052 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f55f65b-38d0-4ab1-91b7-7b64c7fe8170 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:19.625032 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f55f65b-38d0-4ab1-91b7-7b64c7fe8170 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:19.627361 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f55f65b-38d0-4ab1-91b7-7b64c7fe8170 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:19.643842 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1226/2451] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:36:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:36:19.942107 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8149a9d0-cb27-4f6c-9df0-e49bdf840ac4 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:19.943697 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8149a9d0-cb27-4f6c-9df0-e49bdf840ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:19.958883 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:19.964851 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8149a9d0-cb27-4f6c-9df0-e49bdf840ac4 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:19.975012 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd4215c-e4e8-45b6-97c8-eeec48bb818c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:19.975748 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4215c-e4e8-45b6-97c8-eeec48bb818c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:19.976089 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4215c-e4e8-45b6-97c8-eeec48bb818c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:19.976421 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4215c-e4e8-45b6-97c8-eeec48bb818c admin admin] PATH_INFO: `/v3/projects/feb60e9ea1de4b5382ee941d4d5f5454` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:19.977828 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd4215c-e4e8-45b6-97c8-eeec48bb818c admin admin] RBAC: Authorizing `identity:delete_project(project_id=feb60e9ea1de4b5382ee941d4d5f5454)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:19.979927 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd4215c-e4e8-45b6-97c8-eeec48bb818c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:20.033396 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1fd4215c-e4e8-45b6-97c8-eeec48bb818c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'feb60e9ea1de4b5382ee941d4d5f5454', 'request_id': 'req-1fd4215c-e4e8-45b6-97c8-eeec48bb818c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:20.041715 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1fd4215c-e4e8-45b6-97c8-eeec48bb818c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'feb60e9ea1de4b5382ee941d4d5f5454', 'request_id': 'req-1fd4215c-e4e8-45b6-97c8-eeec48bb818c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:20.049281 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1fd4215c-e4e8-45b6-97c8-eeec48bb818c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'feb60e9ea1de4b5382ee941d4d5f5454', 'request_id': 'req-1fd4215c-e4e8-45b6-97c8-eeec48bb818c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:20.061259 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1fd4215c-e4e8-45b6-97c8-eeec48bb818c admin admin] The token cache is being invalidate because project feb60e9ea1de4b5382ee941d4d5f5454 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:36:20.061548 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1fd4215c-e4e8-45b6-97c8-eeec48bb818c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:20.062284 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1fd4215c-e4e8-45b6-97c8-eeec48bb818c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:20.063947 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1226/2452] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:36:19 2022] DELETE /identity/v3/projects/feb60e9ea1de4b5382ee941d4d5f5454 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:20.069020 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9132d7a6-258e-4db4-b6b4-b3a7235cae3a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:20.069314 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9132d7a6-258e-4db4-b6b4-b3a7235cae3a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:20.069588 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9132d7a6-258e-4db4-b6b4-b3a7235cae3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:20.090944 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-9132d7a6-258e-4db4-b6b4-b3a7235cae3a None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:20.092167 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9132d7a6-258e-4db4-b6b4-b3a7235cae3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:20.156801 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1227/2453] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:20.165868 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5470e4-2bde-4c57-9a6a-0776e0e1e17a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:20.166167 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5470e4-2bde-4c57-9a6a-0776e0e1e17a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:20.166454 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5470e4-2bde-4c57-9a6a-0776e0e1e17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:20.187858 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-bf5470e4-2bde-4c57-9a6a-0776e0e1e17a None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:20.189054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bf5470e4-2bde-4c57-9a6a-0776e0e1e17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:20.196965 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1227/2454] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:20.201359 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9132d7a6-258e-4db4-b6b4-b3a7235cae3a None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:20.211265 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9132d7a6-258e-4db4-b6b4-b3a7235cae3a None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:20.215971 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f9faa1-9fd2-465e-84c7-0a1105e9dcef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:20.216582 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9faa1-9fd2-465e-84c7-0a1105e9dcef admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:20.216859 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9faa1-9fd2-465e-84c7-0a1105e9dcef admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:20.217149 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9faa1-9fd2-465e-84c7-0a1105e9dcef admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:20.217823 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f9faa1-9fd2-465e-84c7-0a1105e9dcef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:36:20.218121 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f9faa1-9fd2-465e-84c7-0a1105e9dcef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:20.219740 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f9faa1-9fd2-465e-84c7-0a1105e9dcef admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:20.231213 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1228/2455] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:36:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:36:20.235938 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5470e4-2bde-4c57-9a6a-0776e0e1e17a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:20.245410 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5470e4-2bde-4c57-9a6a-0776e0e1e17a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:20.250978 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc8ee3f-b12e-4eb3-83f0-2a4bb8e9bd25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:20.251750 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc8ee3f-b12e-4eb3-83f0-2a4bb8e9bd25 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:20.252026 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc8ee3f-b12e-4eb3-83f0-2a4bb8e9bd25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:20.252296 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc8ee3f-b12e-4eb3-83f0-2a4bb8e9bd25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:20.252952 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc8ee3f-b12e-4eb3-83f0-2a4bb8e9bd25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:20.254857 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc8ee3f-b12e-4eb3-83f0-2a4bb8e9bd25 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:20.287544 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1228/2456] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:36:20 2022] POST /identity/v3/projects => generated 365 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:36:20.292411 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f9faa1-9fd2-465e-84c7-0a1105e9dcef admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:20.307953 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f9faa1-9fd2-465e-84c7-0a1105e9dcef admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:20.333780 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:20.366467 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d060840-c746-420e-b88d-3dc24c89119a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:20.367119 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d060840-c746-420e-b88d-3dc24c89119a admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:20.367402 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d060840-c746-420e-b88d-3dc24c89119a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:20.367672 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d060840-c746-420e-b88d-3dc24c89119a admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:20.368346 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d060840-c746-420e-b88d-3dc24c89119a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:20.370271 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d060840-c746-420e-b88d-3dc24c89119a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:20.394761 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1229/2457] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:36:20 2022] POST /identity/v3/users => generated 283 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:36:20.398873 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc8ee3f-b12e-4eb3-83f0-2a4bb8e9bd25 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:20.409806 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc8ee3f-b12e-4eb3-83f0-2a4bb8e9bd25 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:20.415581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d29b6a-5a5d-4196-b3e5-c0a482959c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:20.416281 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d29b6a-5a5d-4196-b3e5-c0a482959c58 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:20.416502 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d29b6a-5a5d-4196-b3e5-c0a482959c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:20.416804 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d29b6a-5a5d-4196-b3e5-c0a482959c58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:20.417518 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d29b6a-5a5d-4196-b3e5-c0a482959c58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:20.419260 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d29b6a-5a5d-4196-b3e5-c0a482959c58 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:20.429591 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1229/2458] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:36:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:36:20.433640 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d060840-c746-420e-b88d-3dc24c89119a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:20.443606 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d060840-c746-420e-b88d-3dc24c89119a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:20.448516 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b7421e-8623-4c2d-8b5c-2ce43ee4aca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:20.449113 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b7421e-8623-4c2d-8b5c-2ce43ee4aca9 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:20.449403 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b7421e-8623-4c2d-8b5c-2ce43ee4aca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:20.449752 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b7421e-8623-4c2d-8b5c-2ce43ee4aca9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:20.450454 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b7421e-8623-4c2d-8b5c-2ce43ee4aca9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:20.452086 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b7421e-8623-4c2d-8b5c-2ce43ee4aca9 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:20.462671 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1230/2459] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:36:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:36:20.467237 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d29b6a-5a5d-4196-b3e5-c0a482959c58 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:20.478298 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d29b6a-5a5d-4196-b3e5-c0a482959c58 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:20.483694 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956ebebd-c1f3-44c5-b718-c02a3690d7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:20.484354 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ebebd-c1f3-44c5-b718-c02a3690d7ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:20.484745 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ebebd-c1f3-44c5-b718-c02a3690d7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:20.485017 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ebebd-c1f3-44c5-b718-c02a3690d7ca admin admin] PATH_INFO: `/v3/projects/415651d9667b4da3876031cf25a629d9/users/9a874bdda9d943c5824c47fd5db87e7f/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:20.499754 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956ebebd-c1f3-44c5-b718-c02a3690d7ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=415651d9667b4da3876031cf25a629d9, user_id=9a874bdda9d943c5824c47fd5db87e7f, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:20.502427 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956ebebd-c1f3-44c5-b718-c02a3690d7ca admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:20.518488 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1230/2460] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:36:20 2022] PUT /identity/v3/projects/415651d9667b4da3876031cf25a629d9/users/9a874bdda9d943c5824c47fd5db87e7f/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:20.530706 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5880ba-b2d1-4729-8c03-f70733eaf5da None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:20.531057 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5880ba-b2d1-4729-8c03-f70733eaf5da None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:20.531644 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5880ba-b2d1-4729-8c03-f70733eaf5da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:20.567574 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-fb5880ba-b2d1-4729-8c03-f70733eaf5da None None] MFA Rules not processed for user `9a874bdda9d943c5824c47fd5db87e7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:20.569417 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fb5880ba-b2d1-4729-8c03-f70733eaf5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:20.636402 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1231/2461] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:20.642540 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956ebebd-c1f3-44c5-b718-c02a3690d7ca admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:20.726781 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956ebebd-c1f3-44c5-b718-c02a3690d7ca admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:20.732193 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0255e2-9ad4-46ca-948f-d1211252740b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:20.732814 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0255e2-9ad4-46ca-948f-d1211252740b admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:20.733127 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0255e2-9ad4-46ca-948f-d1211252740b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:20.733396 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0255e2-9ad4-46ca-948f-d1211252740b admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:20.734032 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0255e2-9ad4-46ca-948f-d1211252740b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:20.736050 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0255e2-9ad4-46ca-948f-d1211252740b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:20.771354 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1231/2462] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:36:20 2022] POST /identity/v3/projects => generated 363 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:36:20.776193 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5880ba-b2d1-4729-8c03-f70733eaf5da None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:20.788126 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5880ba-b2d1-4729-8c03-f70733eaf5da None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:20.847894 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feca71dc-0dfa-40a3-b591-f72237a58487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:20.848510 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feca71dc-0dfa-40a3-b591-f72237a58487 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:20.848790 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feca71dc-0dfa-40a3-b591-f72237a58487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:20.849063 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feca71dc-0dfa-40a3-b591-f72237a58487 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:20.849713 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feca71dc-0dfa-40a3-b591-f72237a58487 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:20.851662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feca71dc-0dfa-40a3-b591-f72237a58487 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:20.875681 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1232/2463] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:36:20 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:36:20.879891 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0255e2-9ad4-46ca-948f-d1211252740b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:20.890948 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0255e2-9ad4-46ca-948f-d1211252740b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:20.895831 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1ea67b-59a0-40cd-a50b-c0c00f4b522d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:20.896441 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ea67b-59a0-40cd-a50b-c0c00f4b522d admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:20.896723 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ea67b-59a0-40cd-a50b-c0c00f4b522d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:20.896998 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ea67b-59a0-40cd-a50b-c0c00f4b522d admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:20.897687 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1ea67b-59a0-40cd-a50b-c0c00f4b522d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:20.899329 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1ea67b-59a0-40cd-a50b-c0c00f4b522d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:20.901341 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:20.911241 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1232/2464] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:36:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:36:20.915402 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feca71dc-0dfa-40a3-b591-f72237a58487 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:20.925707 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feca71dc-0dfa-40a3-b591-f72237a58487 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:20.931335 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd258d8-9d45-4546-a321-a2b52f22bf4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:20.932007 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd258d8-9d45-4546-a321-a2b52f22bf4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:20.932286 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd258d8-9d45-4546-a321-a2b52f22bf4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:20.932556 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd258d8-9d45-4546-a321-a2b52f22bf4f admin admin] PATH_INFO: `/v3/projects/5951ab89465f4503a7e97cb50bb63a01/users/a64d24d976b94d6b98915d8ee72647b4/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:20.948563 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd258d8-9d45-4546-a321-a2b52f22bf4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5951ab89465f4503a7e97cb50bb63a01, user_id=a64d24d976b94d6b98915d8ee72647b4, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:20.951373 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd258d8-9d45-4546-a321-a2b52f22bf4f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:20.962604 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:20.968350 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1233/2465] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:36:20 2022] PUT /identity/v3/projects/5951ab89465f4503a7e97cb50bb63a01/users/a64d24d976b94d6b98915d8ee72647b4/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:20.972545 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1ea67b-59a0-40cd-a50b-c0c00f4b522d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:20.984041 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1ea67b-59a0-40cd-a50b-c0c00f4b522d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:21.049052 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47928d08-9a23-47c5-9612-b1718a176108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:21.049671 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47928d08-9a23-47c5-9612-b1718a176108 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:21.049951 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47928d08-9a23-47c5-9612-b1718a176108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:21.050203 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47928d08-9a23-47c5-9612-b1718a176108 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:21.050926 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47928d08-9a23-47c5-9612-b1718a176108 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:21.052570 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47928d08-9a23-47c5-9612-b1718a176108 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:21.062301 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1233/2466] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:36:20 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:36:21.067391 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd258d8-9d45-4546-a321-a2b52f22bf4f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:21.079126 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd258d8-9d45-4546-a321-a2b52f22bf4f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:21.084543 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407f4c5-c830-469e-ab97-d617a2ed4017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:21.085243 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407f4c5-c830-469e-ab97-d617a2ed4017 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:21.085518 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407f4c5-c830-469e-ab97-d617a2ed4017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:21.085786 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407f4c5-c830-469e-ab97-d617a2ed4017 admin admin] PATH_INFO: `/v3/projects/5951ab89465f4503a7e97cb50bb63a01/users/a64d24d976b94d6b98915d8ee72647b4/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:21.087795 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1407f4c5-c830-469e-ab97-d617a2ed4017 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5951ab89465f4503a7e97cb50bb63a01, user_id=a64d24d976b94d6b98915d8ee72647b4, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:21.090700 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1407f4c5-c830-469e-ab97-d617a2ed4017 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:21.101922 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1234/2467] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:36:21 2022] PUT /identity/v3/projects/5951ab89465f4503a7e97cb50bb63a01/users/a64d24d976b94d6b98915d8ee72647b4/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:21.112399 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20941ca-1ccd-4d8b-89e3-94637d570a91 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:21.112681 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20941ca-1ccd-4d8b-89e3-94637d570a91 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:21.112953 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20941ca-1ccd-4d8b-89e3-94637d570a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:21.141782 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a20941ca-1ccd-4d8b-89e3-94637d570a91 None None] MFA Rules not processed for user `a64d24d976b94d6b98915d8ee72647b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:21.142972 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a20941ca-1ccd-4d8b-89e3-94637d570a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:21.216397 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1234/2468] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:21 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:21.228122 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407f4c5-c830-469e-ab97-d617a2ed4017 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:21.230144 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1407f4c5-c830-469e-ab97-d617a2ed4017 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:21.296100 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407f4c5-c830-469e-ab97-d617a2ed4017 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:21.300077 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101edcd-abca-4c53-a0c2-a585fee4f691 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:21.300710 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101edcd-abca-4c53-a0c2-a585fee4f691 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:21.300990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101edcd-abca-4c53-a0c2-a585fee4f691 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:21.301261 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101edcd-abca-4c53-a0c2-a585fee4f691 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:21.310958 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101edcd-abca-4c53-a0c2-a585fee4f691 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:21.312406 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101edcd-abca-4c53-a0c2-a585fee4f691 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:21.321789 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1235/2469] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:36:21 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:36:21.787654 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20941ca-1ccd-4d8b-89e3-94637d570a91 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:21.863179 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20941ca-1ccd-4d8b-89e3-94637d570a91 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:21.868254 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c1520-d12a-48d3-b128-081477a144d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:21.868904 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c1520-d12a-48d3-b128-081477a144d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:21.869186 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c1520-d12a-48d3-b128-081477a144d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:21.869459 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c1520-d12a-48d3-b128-081477a144d1 admin admin] PATH_INFO: `/v3/users/9a874bdda9d943c5824c47fd5db87e7f` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:21.870819 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307c1520-d12a-48d3-b128-081477a144d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a874bdda9d943c5824c47fd5db87e7f)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:21.873226 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307c1520-d12a-48d3-b128-081477a144d1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:21.902079 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:21.932719 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-307c1520-d12a-48d3-b128-081477a144d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a874bdda9d943c5824c47fd5db87e7f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:21.933407 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-307c1520-d12a-48d3-b128-081477a144d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a874bdda9d943c5824c47fd5db87e7f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:21.934043 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-307c1520-d12a-48d3-b128-081477a144d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a874bdda9d943c5824c47fd5db87e7f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:21.943256 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-307c1520-d12a-48d3-b128-081477a144d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a874bdda9d943c5824c47fd5db87e7f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:21.952390 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-307c1520-d12a-48d3-b128-081477a144d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a874bdda9d943c5824c47fd5db87e7f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:21.970445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1235/2470] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:36:21 2022] DELETE /identity/v3/users/9a874bdda9d943c5824c47fd5db87e7f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:21.975262 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4ce6f-4d64-4652-b326-6613595245ab None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:21.975605 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4ce6f-4d64-4652-b326-6613595245ab None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:21.975883 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4ce6f-4d64-4652-b326-6613595245ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:21.995292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-72a4ce6f-4d64-4652-b326-6613595245ab None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:21.996562 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-72a4ce6f-4d64-4652-b326-6613595245ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:22.060323 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1236/2471] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:22.070409 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c1520-d12a-48d3-b128-081477a144d1 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:22.071909 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-307c1520-d12a-48d3-b128-081477a144d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:22.139379 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c1520-d12a-48d3-b128-081477a144d1 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:22.143839 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34556c7-9a8e-4fed-82f9-80ec3dad7954 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:22.144471 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34556c7-9a8e-4fed-82f9-80ec3dad7954 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:22.144743 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34556c7-9a8e-4fed-82f9-80ec3dad7954 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:22.145036 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34556c7-9a8e-4fed-82f9-80ec3dad7954 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:22.166546 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34556c7-9a8e-4fed-82f9-80ec3dad7954 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:22.168062 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34556c7-9a8e-4fed-82f9-80ec3dad7954 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:22.178826 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1236/2472] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:36:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:36:22.460555 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a4ce6f-4d64-4652-b326-6613595245ab None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:22.462788 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-72a4ce6f-4d64-4652-b326-6613595245ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:22.477024 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a4ce6f-4d64-4652-b326-6613595245ab None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:22.482645 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bb6e53-f268-4ac1-abee-ee3d579959fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:22.483679 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb6e53-f268-4ac1-abee-ee3d579959fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:22.484179 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb6e53-f268-4ac1-abee-ee3d579959fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:22.484659 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb6e53-f268-4ac1-abee-ee3d579959fb admin admin] PATH_INFO: `/v3/projects/415651d9667b4da3876031cf25a629d9` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:22.487407 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bb6e53-f268-4ac1-abee-ee3d579959fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=415651d9667b4da3876031cf25a629d9)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:22.491412 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bb6e53-f268-4ac1-abee-ee3d579959fb admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:22.545727 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f3bb6e53-f268-4ac1-abee-ee3d579959fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '415651d9667b4da3876031cf25a629d9', 'request_id': 'req-f3bb6e53-f268-4ac1-abee-ee3d579959fb'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:22.553995 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f3bb6e53-f268-4ac1-abee-ee3d579959fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '415651d9667b4da3876031cf25a629d9', 'request_id': 'req-f3bb6e53-f268-4ac1-abee-ee3d579959fb'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:22.564143 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f3bb6e53-f268-4ac1-abee-ee3d579959fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '415651d9667b4da3876031cf25a629d9', 'request_id': 'req-f3bb6e53-f268-4ac1-abee-ee3d579959fb'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:22.584716 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f3bb6e53-f268-4ac1-abee-ee3d579959fb admin admin] The token cache is being invalidate because project 415651d9667b4da3876031cf25a629d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:36:22.584998 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f3bb6e53-f268-4ac1-abee-ee3d579959fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:22.585904 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f3bb6e53-f268-4ac1-abee-ee3d579959fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:22.587946 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1237/2473] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:36:22 2022] DELETE /identity/v3/projects/415651d9667b4da3876031cf25a629d9 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:22.592605 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34556c7-9a8e-4fed-82f9-80ec3dad7954 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:22.603112 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34556c7-9a8e-4fed-82f9-80ec3dad7954 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:22.662833 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6768ec52-980d-4eff-bae1-4648b547e405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:22.663516 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6768ec52-980d-4eff-bae1-4648b547e405 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:22.663800 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6768ec52-980d-4eff-bae1-4648b547e405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:22.664069 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6768ec52-980d-4eff-bae1-4648b547e405 admin admin] PATH_INFO: `/v3/users/a64d24d976b94d6b98915d8ee72647b4` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:22.665268 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6768ec52-980d-4eff-bae1-4648b547e405 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a64d24d976b94d6b98915d8ee72647b4)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:22.667239 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6768ec52-980d-4eff-bae1-4648b547e405 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:22.713089 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:22.716146 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6768ec52-980d-4eff-bae1-4648b547e405 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a64d24d976b94d6b98915d8ee72647b4'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:22.717256 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6768ec52-980d-4eff-bae1-4648b547e405 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a64d24d976b94d6b98915d8ee72647b4'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:22.718086 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6768ec52-980d-4eff-bae1-4648b547e405 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a64d24d976b94d6b98915d8ee72647b4'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:22.727785 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6768ec52-980d-4eff-bae1-4648b547e405 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a64d24d976b94d6b98915d8ee72647b4'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:22.738572 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6768ec52-980d-4eff-bae1-4648b547e405 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a64d24d976b94d6b98915d8ee72647b4'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:22.756574 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1237/2474] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:36:22 2022] DELETE /identity/v3/users/a64d24d976b94d6b98915d8ee72647b4 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:23.110293 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bb6e53-f268-4ac1-abee-ee3d579959fb admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:23.111968 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f3bb6e53-f268-4ac1-abee-ee3d579959fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:23.126456 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bb6e53-f268-4ac1-abee-ee3d579959fb admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:23.139793 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:23.194119 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc123b1-1787-45fa-ad6e-b7f81d11bf89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:23.194917 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc123b1-1787-45fa-ad6e-b7f81d11bf89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:23.195200 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc123b1-1787-45fa-ad6e-b7f81d11bf89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:23.195476 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc123b1-1787-45fa-ad6e-b7f81d11bf89 admin admin] PATH_INFO: `/v3/projects/5951ab89465f4503a7e97cb50bb63a01` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:23.196691 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc123b1-1787-45fa-ad6e-b7f81d11bf89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5951ab89465f4503a7e97cb50bb63a01)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:23.198615 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc123b1-1787-45fa-ad6e-b7f81d11bf89 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:23.247917 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3dc123b1-1787-45fa-ad6e-b7f81d11bf89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5951ab89465f4503a7e97cb50bb63a01', 'request_id': 'req-3dc123b1-1787-45fa-ad6e-b7f81d11bf89'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:23.254167 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3dc123b1-1787-45fa-ad6e-b7f81d11bf89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5951ab89465f4503a7e97cb50bb63a01', 'request_id': 'req-3dc123b1-1787-45fa-ad6e-b7f81d11bf89'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:23.263805 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3dc123b1-1787-45fa-ad6e-b7f81d11bf89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5951ab89465f4503a7e97cb50bb63a01', 'request_id': 'req-3dc123b1-1787-45fa-ad6e-b7f81d11bf89'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:23.275822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3dc123b1-1787-45fa-ad6e-b7f81d11bf89 admin admin] The token cache is being invalidate because project 5951ab89465f4503a7e97cb50bb63a01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:36:23.276110 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3dc123b1-1787-45fa-ad6e-b7f81d11bf89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:23.276815 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3dc123b1-1787-45fa-ad6e-b7f81d11bf89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:23.282300 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1238/2475] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:36:23 2022] DELETE /identity/v3/projects/5951ab89465f4503a7e97cb50bb63a01 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:23.284177 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5068d-2860-4df6-a400-6b4db4df43f6 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:23.284676 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5068d-2860-4df6-a400-6b4db4df43f6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:23.284893 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5068d-2860-4df6-a400-6b4db4df43f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:23.305727 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b2b5068d-2860-4df6-a400-6b4db4df43f6 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:23.306959 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b2b5068d-2860-4df6-a400-6b4db4df43f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:23.380909 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1238/2476] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:36:23.394790 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5853ee-ab64-4f06-af03-ea04d1f48adc None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:23.395152 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5853ee-ab64-4f06-af03-ea04d1f48adc None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:23.395385 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5853ee-ab64-4f06-af03-ea04d1f48adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:23.419862 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-3f5853ee-ab64-4f06-af03-ea04d1f48adc None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:23.421102 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3f5853ee-ab64-4f06-af03-ea04d1f48adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:23.430192 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1239/2477] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:23.434537 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5068d-2860-4df6-a400-6b4db4df43f6 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:23.444894 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5068d-2860-4df6-a400-6b4db4df43f6 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:23.449927 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b903866c-0cdd-4e84-b5ba-03af70b563cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:23.450566 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903866c-0cdd-4e84-b5ba-03af70b563cb admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:23.450848 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903866c-0cdd-4e84-b5ba-03af70b563cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:23.451172 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903866c-0cdd-4e84-b5ba-03af70b563cb admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:23.451889 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b903866c-0cdd-4e84-b5ba-03af70b563cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:36:23.452198 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b903866c-0cdd-4e84-b5ba-03af70b563cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:23.453810 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b903866c-0cdd-4e84-b5ba-03af70b563cb admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:23.466939 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1239/2478] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:36:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:36:23.471403 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5853ee-ab64-4f06-af03-ea04d1f48adc None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:23.481440 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5853ee-ab64-4f06-af03-ea04d1f48adc None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:23.486123 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2011d788-e037-43c9-a749-4f0a6f305d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:23.486747 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2011d788-e037-43c9-a749-4f0a6f305d23 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:23.487055 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2011d788-e037-43c9-a749-4f0a6f305d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:23.487349 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2011d788-e037-43c9-a749-4f0a6f305d23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:23.488028 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2011d788-e037-43c9-a749-4f0a6f305d23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:23.489927 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2011d788-e037-43c9-a749-4f0a6f305d23 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:23.521405 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1240/2479] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:36:23 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:36:23.525827 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b903866c-0cdd-4e84-b5ba-03af70b563cb admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:23.537195 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b903866c-0cdd-4e84-b5ba-03af70b563cb admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:23.604110 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e7e93b-b072-46ef-a359-0168998601fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:23.604935 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e7e93b-b072-46ef-a359-0168998601fd admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:23.605216 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e7e93b-b072-46ef-a359-0168998601fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:23.605485 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e7e93b-b072-46ef-a359-0168998601fd admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:23.606244 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e7e93b-b072-46ef-a359-0168998601fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:23.608212 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e7e93b-b072-46ef-a359-0168998601fd admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:23.633975 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1240/2480] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:36:23 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:36:23.638589 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2011d788-e037-43c9-a749-4f0a6f305d23 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:23.649699 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2011d788-e037-43c9-a749-4f0a6f305d23 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:23.656692 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7554456f-d4fb-4f85-b57d-e9963eb79290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:23.657577 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7554456f-d4fb-4f85-b57d-e9963eb79290 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:23.657860 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7554456f-d4fb-4f85-b57d-e9963eb79290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:23.658137 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7554456f-d4fb-4f85-b57d-e9963eb79290 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:23.658849 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7554456f-d4fb-4f85-b57d-e9963eb79290 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:23.660499 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7554456f-d4fb-4f85-b57d-e9963eb79290 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:23.672518 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1241/2481] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:36:23.676447 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e7e93b-b072-46ef-a359-0168998601fd admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:23.687616 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e7e93b-b072-46ef-a359-0168998601fd admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:23.693982 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42458edd-19c7-493b-9d2f-e488167e0e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:23.694772 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42458edd-19c7-493b-9d2f-e488167e0e25 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:23.695227 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42458edd-19c7-493b-9d2f-e488167e0e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:23.697115 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42458edd-19c7-493b-9d2f-e488167e0e25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:23.697905 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42458edd-19c7-493b-9d2f-e488167e0e25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:23.699759 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42458edd-19c7-493b-9d2f-e488167e0e25 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:23.711703 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1241/2482] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:36:23.716260 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7554456f-d4fb-4f85-b57d-e9963eb79290 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:23.727636 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7554456f-d4fb-4f85-b57d-e9963eb79290 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:23.733294 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b09b9d-aa33-4486-b0aa-01d3b23abcc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:23.734003 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b09b9d-aa33-4486-b0aa-01d3b23abcc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:23.734314 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b09b9d-aa33-4486-b0aa-01d3b23abcc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:23.734597 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b09b9d-aa33-4486-b0aa-01d3b23abcc1 admin admin] PATH_INFO: `/v3/projects/d0198eadc79f47ad8388c351f86f0362/users/60a2d254327f450383dba062e39e6e03/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:23.751040 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b09b9d-aa33-4486-b0aa-01d3b23abcc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0198eadc79f47ad8388c351f86f0362, user_id=60a2d254327f450383dba062e39e6e03, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:23.753715 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b09b9d-aa33-4486-b0aa-01d3b23abcc1 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:23.779162 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1242/2483] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:36:23 2022] PUT /identity/v3/projects/d0198eadc79f47ad8388c351f86f0362/users/60a2d254327f450383dba062e39e6e03/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:23.791383 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290065c3-1589-4c2b-bf8f-2ce1d052c4ac None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:23.791693 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290065c3-1589-4c2b-bf8f-2ce1d052c4ac None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:23.792101 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290065c3-1589-4c2b-bf8f-2ce1d052c4ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:23.822284 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-290065c3-1589-4c2b-bf8f-2ce1d052c4ac None None] MFA Rules not processed for user `60a2d254327f450383dba062e39e6e03`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:23.823622 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-290065c3-1589-4c2b-bf8f-2ce1d052c4ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:23.838061 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:23.907806 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1242/2484] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:36:23.912381 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b09b9d-aa33-4486-b0aa-01d3b23abcc1 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:23.972631 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:23.997256 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b09b9d-aa33-4486-b0aa-01d3b23abcc1 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:24.002130 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bc5081-e421-4dc8-9760-e7ee39a89a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:24.002759 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc5081-e421-4dc8-9760-e7ee39a89a74 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:24.003080 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc5081-e421-4dc8-9760-e7ee39a89a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:24.003356 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc5081-e421-4dc8-9760-e7ee39a89a74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:24.004087 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bc5081-e421-4dc8-9760-e7ee39a89a74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:24.006013 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bc5081-e421-4dc8-9760-e7ee39a89a74 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:24.035809 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1243/2485] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:36:23 2022] POST /identity/v3/projects => generated 385 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:36:24.040065 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290065c3-1589-4c2b-bf8f-2ce1d052c4ac None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:24.051301 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290065c3-1589-4c2b-bf8f-2ce1d052c4ac None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:24.111360 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e98f5-bb54-4665-a112-337430de706c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:24.111992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e98f5-bb54-4665-a112-337430de706c admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:24.112367 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e98f5-bb54-4665-a112-337430de706c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:24.112673 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e98f5-bb54-4665-a112-337430de706c admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:24.113313 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397e98f5-bb54-4665-a112-337430de706c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:24.115360 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397e98f5-bb54-4665-a112-337430de706c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:24.139455 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1243/2486] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:36:24 2022] POST /identity/v3/users => generated 299 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:36:24.143813 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bc5081-e421-4dc8-9760-e7ee39a89a74 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:24.164254 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bc5081-e421-4dc8-9760-e7ee39a89a74 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:24.169712 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5177d09-bef5-4385-b766-aa67970fb290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:24.170449 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5177d09-bef5-4385-b766-aa67970fb290 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:24.170794 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5177d09-bef5-4385-b766-aa67970fb290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:24.171167 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5177d09-bef5-4385-b766-aa67970fb290 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:24.171881 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5177d09-bef5-4385-b766-aa67970fb290 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:24.173504 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5177d09-bef5-4385-b766-aa67970fb290 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:24.185178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1244/2487] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:36:24.189888 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e98f5-bb54-4665-a112-337430de706c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:24.202339 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e98f5-bb54-4665-a112-337430de706c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:24.208022 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57339a2c-672e-4776-be5e-0c77a32de282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:24.208756 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57339a2c-672e-4776-be5e-0c77a32de282 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:24.209031 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57339a2c-672e-4776-be5e-0c77a32de282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:24.209301 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57339a2c-672e-4776-be5e-0c77a32de282 admin admin] PATH_INFO: `/v3/projects/392880261e8e41eb8669ce0984c0a2aa/users/5f285f97504a4070bfbf54c6a6292476/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:24.227602 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57339a2c-672e-4776-be5e-0c77a32de282 admin admin] RBAC: Authorizing `identity:create_grant(project_id=392880261e8e41eb8669ce0984c0a2aa, user_id=5f285f97504a4070bfbf54c6a6292476, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:24.230380 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57339a2c-672e-4776-be5e-0c77a32de282 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:24.246302 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1244/2488] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:36:24 2022] PUT /identity/v3/projects/392880261e8e41eb8669ce0984c0a2aa/users/5f285f97504a4070bfbf54c6a6292476/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:24.250512 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5177d09-bef5-4385-b766-aa67970fb290 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:24.260560 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5177d09-bef5-4385-b766-aa67970fb290 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:24.320278 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9788ec24-0575-429a-b73f-4d0c94e88ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:24.320985 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788ec24-0575-429a-b73f-4d0c94e88ce9 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:24.321267 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788ec24-0575-429a-b73f-4d0c94e88ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:24.321541 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788ec24-0575-429a-b73f-4d0c94e88ce9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:24.322241 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9788ec24-0575-429a-b73f-4d0c94e88ce9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:24.323872 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9788ec24-0575-429a-b73f-4d0c94e88ce9 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:24.333902 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1245/2489] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:36:24.338488 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57339a2c-672e-4776-be5e-0c77a32de282 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:24.349391 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57339a2c-672e-4776-be5e-0c77a32de282 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:24.354525 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d061ef-968b-4197-8c8c-06ab6d17e334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:24.355282 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d061ef-968b-4197-8c8c-06ab6d17e334 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:24.355802 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d061ef-968b-4197-8c8c-06ab6d17e334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:24.356033 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d061ef-968b-4197-8c8c-06ab6d17e334 admin admin] PATH_INFO: `/v3/projects/392880261e8e41eb8669ce0984c0a2aa/users/5f285f97504a4070bfbf54c6a6292476/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:24.359689 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d061ef-968b-4197-8c8c-06ab6d17e334 admin admin] RBAC: Authorizing `identity:create_grant(project_id=392880261e8e41eb8669ce0984c0a2aa, user_id=5f285f97504a4070bfbf54c6a6292476, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:24.361183 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d061ef-968b-4197-8c8c-06ab6d17e334 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:24.372940 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1245/2490] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:36:24 2022] PUT /identity/v3/projects/392880261e8e41eb8669ce0984c0a2aa/users/5f285f97504a4070bfbf54c6a6292476/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:36:24.383352 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f03ec28-f86e-4d91-91f3-513f8e753cd5 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:24.383640 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f03ec28-f86e-4d91-91f3-513f8e753cd5 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:24.383925 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f03ec28-f86e-4d91-91f3-513f8e753cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:24.416131 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-6f03ec28-f86e-4d91-91f3-513f8e753cd5 None None] MFA Rules not processed for user `5f285f97504a4070bfbf54c6a6292476`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:24.417434 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6f03ec28-f86e-4d91-91f3-513f8e753cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:24.484586 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1246/2491] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:24 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:24.496436 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d061ef-968b-4197-8c8c-06ab6d17e334 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:24.498659 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b0d061ef-968b-4197-8c8c-06ab6d17e334 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:24.559407 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d061ef-968b-4197-8c8c-06ab6d17e334 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:24.564011 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1575e7f2-be73-43ed-b4d1-3ed395859b8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:24.564636 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1575e7f2-be73-43ed-b4d1-3ed395859b8f service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:24.564917 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1575e7f2-be73-43ed-b4d1-3ed395859b8f service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:24.565185 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1575e7f2-be73-43ed-b4d1-3ed395859b8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:24.575330 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1575e7f2-be73-43ed-b4d1-3ed395859b8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:24.576780 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1575e7f2-be73-43ed-b4d1-3ed395859b8f service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:24.586104 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1246/2492] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:36:24 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:36:24.615421 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f03ec28-f86e-4d91-91f3-513f8e753cd5 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:24.657190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:24.682355 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f03ec28-f86e-4d91-91f3-513f8e753cd5 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:24.687970 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68513269-418a-4c70-b97e-df752b2c45bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:24.688628 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68513269-418a-4c70-b97e-df752b2c45bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:24.688908 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68513269-418a-4c70-b97e-df752b2c45bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:24.689218 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68513269-418a-4c70-b97e-df752b2c45bd admin admin] PATH_INFO: `/v3/users/60a2d254327f450383dba062e39e6e03` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:24.690524 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68513269-418a-4c70-b97e-df752b2c45bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=60a2d254327f450383dba062e39e6e03)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:24.692640 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68513269-418a-4c70-b97e-df752b2c45bd admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:24.748034 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-68513269-418a-4c70-b97e-df752b2c45bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60a2d254327f450383dba062e39e6e03'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:24.756721 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-68513269-418a-4c70-b97e-df752b2c45bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60a2d254327f450383dba062e39e6e03'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:24.757352 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-68513269-418a-4c70-b97e-df752b2c45bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60a2d254327f450383dba062e39e6e03'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:24.771316 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-68513269-418a-4c70-b97e-df752b2c45bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60a2d254327f450383dba062e39e6e03'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:24.771970 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-68513269-418a-4c70-b97e-df752b2c45bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60a2d254327f450383dba062e39e6e03'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:24.784634 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1247/2493] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:36:24 2022] DELETE /identity/v3/users/60a2d254327f450383dba062e39e6e03 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:24.790771 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd4fd1d-8b19-4baa-a611-426581dd3154 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:24.791078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd4fd1d-8b19-4baa-a611-426581dd3154 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:24.791354 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd4fd1d-8b19-4baa-a611-426581dd3154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:24.813805 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-7cd4fd1d-8b19-4baa-a611-426581dd3154 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:24.815069 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7cd4fd1d-8b19-4baa-a611-426581dd3154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:24.820490 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:24.881782 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1247/2494] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:24.892289 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68513269-418a-4c70-b97e-df752b2c45bd admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:24.893828 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-68513269-418a-4c70-b97e-df752b2c45bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:24.954552 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68513269-418a-4c70-b97e-df752b2c45bd admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:24.958495 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1465af-ee52-4f56-8a7a-271ed7de233f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:24.959163 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1465af-ee52-4f56-8a7a-271ed7de233f service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:24.959457 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1465af-ee52-4f56-8a7a-271ed7de233f service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:24.959750 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1465af-ee52-4f56-8a7a-271ed7de233f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:24.971173 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1465af-ee52-4f56-8a7a-271ed7de233f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:24.972684 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1465af-ee52-4f56-8a7a-271ed7de233f service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:24.983400 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1248/2495] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:36:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:36:25.276890 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd4fd1d-8b19-4baa-a611-426581dd3154 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:25.278723 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7cd4fd1d-8b19-4baa-a611-426581dd3154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:25.290937 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd4fd1d-8b19-4baa-a611-426581dd3154 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:25.297275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4568645-1a3a-4479-8c19-6042f3a502fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:25.298049 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4568645-1a3a-4479-8c19-6042f3a502fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:25.298613 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4568645-1a3a-4479-8c19-6042f3a502fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:25.298748 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4568645-1a3a-4479-8c19-6042f3a502fb admin admin] PATH_INFO: `/v3/projects/d0198eadc79f47ad8388c351f86f0362` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:25.300085 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4568645-1a3a-4479-8c19-6042f3a502fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0198eadc79f47ad8388c351f86f0362)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:25.302153 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4568645-1a3a-4479-8c19-6042f3a502fb admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:25.355367 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d4568645-1a3a-4479-8c19-6042f3a502fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0198eadc79f47ad8388c351f86f0362', 'request_id': 'req-d4568645-1a3a-4479-8c19-6042f3a502fb'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:25.365419 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d4568645-1a3a-4479-8c19-6042f3a502fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0198eadc79f47ad8388c351f86f0362', 'request_id': 'req-d4568645-1a3a-4479-8c19-6042f3a502fb'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:25.371972 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d4568645-1a3a-4479-8c19-6042f3a502fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0198eadc79f47ad8388c351f86f0362', 'request_id': 'req-d4568645-1a3a-4479-8c19-6042f3a502fb'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:25.383240 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d4568645-1a3a-4479-8c19-6042f3a502fb admin admin] The token cache is being invalidate because project d0198eadc79f47ad8388c351f86f0362 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:36:25.383530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d4568645-1a3a-4479-8c19-6042f3a502fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:25.384160 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d4568645-1a3a-4479-8c19-6042f3a502fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:25.385750 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1248/2496] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:36:25 2022] DELETE /identity/v3/projects/d0198eadc79f47ad8388c351f86f0362 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:25.390477 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1465af-ee52-4f56-8a7a-271ed7de233f service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:25.400756 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1465af-ee52-4f56-8a7a-271ed7de233f service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:25.454647 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:25.461775 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674ee784-fac8-4092-b1c9-0b5052f69b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:25.462446 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674ee784-fac8-4092-b1c9-0b5052f69b0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:25.463219 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674ee784-fac8-4092-b1c9-0b5052f69b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:25.463759 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674ee784-fac8-4092-b1c9-0b5052f69b0c admin admin] PATH_INFO: `/v3/users/5f285f97504a4070bfbf54c6a6292476` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:25.465696 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674ee784-fac8-4092-b1c9-0b5052f69b0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f285f97504a4070bfbf54c6a6292476)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:25.467904 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674ee784-fac8-4092-b1c9-0b5052f69b0c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:25.528876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-674ee784-fac8-4092-b1c9-0b5052f69b0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f285f97504a4070bfbf54c6a6292476'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:25.537337 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-674ee784-fac8-4092-b1c9-0b5052f69b0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f285f97504a4070bfbf54c6a6292476'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:25.537893 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-674ee784-fac8-4092-b1c9-0b5052f69b0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f285f97504a4070bfbf54c6a6292476'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:25.549974 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-674ee784-fac8-4092-b1c9-0b5052f69b0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f285f97504a4070bfbf54c6a6292476'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:25.550651 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-674ee784-fac8-4092-b1c9-0b5052f69b0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f285f97504a4070bfbf54c6a6292476'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:25.563085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1249/2497] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:36:25 2022] DELETE /identity/v3/users/5f285f97504a4070bfbf54c6a6292476 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:25.849439 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4568645-1a3a-4479-8c19-6042f3a502fb admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:25.851070 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d4568645-1a3a-4479-8c19-6042f3a502fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:25.864925 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4568645-1a3a-4479-8c19-6042f3a502fb admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:25.927339 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb34a1e8-6e12-49a6-a905-cf793eb463eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:25.927990 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb34a1e8-6e12-49a6-a905-cf793eb463eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:25.928267 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb34a1e8-6e12-49a6-a905-cf793eb463eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:25.928538 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb34a1e8-6e12-49a6-a905-cf793eb463eb admin admin] PATH_INFO: `/v3/projects/392880261e8e41eb8669ce0984c0a2aa` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:25.929728 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb34a1e8-6e12-49a6-a905-cf793eb463eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=392880261e8e41eb8669ce0984c0a2aa)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:25.931756 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb34a1e8-6e12-49a6-a905-cf793eb463eb admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:25.972452 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:25.983555 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fb34a1e8-6e12-49a6-a905-cf793eb463eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '392880261e8e41eb8669ce0984c0a2aa', 'request_id': 'req-fb34a1e8-6e12-49a6-a905-cf793eb463eb'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:25.993150 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fb34a1e8-6e12-49a6-a905-cf793eb463eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '392880261e8e41eb8669ce0984c0a2aa', 'request_id': 'req-fb34a1e8-6e12-49a6-a905-cf793eb463eb'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:26.002381 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fb34a1e8-6e12-49a6-a905-cf793eb463eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '392880261e8e41eb8669ce0984c0a2aa', 'request_id': 'req-fb34a1e8-6e12-49a6-a905-cf793eb463eb'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:26.016129 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fb34a1e8-6e12-49a6-a905-cf793eb463eb admin admin] The token cache is being invalidate because project 392880261e8e41eb8669ce0984c0a2aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:36:26.016442 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fb34a1e8-6e12-49a6-a905-cf793eb463eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:26.017142 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fb34a1e8-6e12-49a6-a905-cf793eb463eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:26.018723 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1249/2498] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:36:25 2022] DELETE /identity/v3/projects/392880261e8e41eb8669ce0984c0a2aa => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:26.023869 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9326e3-0b75-49ae-9868-569b98f4a1fd None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:26.024042 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9326e3-0b75-49ae-9868-569b98f4a1fd None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:26.024263 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9326e3-0b75-49ae-9868-569b98f4a1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:26.045965 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-9d9326e3-0b75-49ae-9868-569b98f4a1fd None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:26.047217 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9d9326e3-0b75-49ae-9868-569b98f4a1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:26.110686 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1250/2499] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:26.118352 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785cb21-b94d-4826-956a-da3dfbe3abeb None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:26.118657 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785cb21-b94d-4826-956a-da3dfbe3abeb None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:26.118926 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785cb21-b94d-4826-956a-da3dfbe3abeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:26.144107 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d785cb21-b94d-4826-956a-da3dfbe3abeb None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:26.145457 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d785cb21-b94d-4826-956a-da3dfbe3abeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:26.156161 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1250/2500] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:26.161659 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9326e3-0b75-49ae-9868-569b98f4a1fd None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:26.173011 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9326e3-0b75-49ae-9868-569b98f4a1fd None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:26.179280 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75d438b-e815-43d3-8011-05afac7c9f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:26.179908 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d438b-e815-43d3-8011-05afac7c9f0f admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:26.180191 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d438b-e815-43d3-8011-05afac7c9f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:26.180458 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d438b-e815-43d3-8011-05afac7c9f0f admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:26.181160 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75d438b-e815-43d3-8011-05afac7c9f0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:36:26.181464 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75d438b-e815-43d3-8011-05afac7c9f0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:26.183152 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75d438b-e815-43d3-8011-05afac7c9f0f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:26.195526 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1251/2501] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:36:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:36:26.200211 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d785cb21-b94d-4826-956a-da3dfbe3abeb None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:26.211238 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d785cb21-b94d-4826-956a-da3dfbe3abeb None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:26.217406 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce53f20-cdb1-4066-a6f4-f7978efa2679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:26.218169 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce53f20-cdb1-4066-a6f4-f7978efa2679 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:26.218452 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce53f20-cdb1-4066-a6f4-f7978efa2679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:26.218721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce53f20-cdb1-4066-a6f4-f7978efa2679 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:26.219461 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce53f20-cdb1-4066-a6f4-f7978efa2679 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:26.221559 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce53f20-cdb1-4066-a6f4-f7978efa2679 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:26.253935 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1251/2502] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:36:26 2022] POST /identity/v3/projects => generated 379 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:36:26.258231 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75d438b-e815-43d3-8011-05afac7c9f0f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:26.269600 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75d438b-e815-43d3-8011-05afac7c9f0f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:26.334773 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d4baf-6462-420a-85dd-b1a0f33d5468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:26.335442 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d4baf-6462-420a-85dd-b1a0f33d5468 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:26.335724 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d4baf-6462-420a-85dd-b1a0f33d5468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:26.335995 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d4baf-6462-420a-85dd-b1a0f33d5468 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:26.336655 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040d4baf-6462-420a-85dd-b1a0f33d5468 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:26.338579 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040d4baf-6462-420a-85dd-b1a0f33d5468 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:26.348484 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:26.363807 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1252/2503] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:36:26 2022] POST /identity/v3/users => generated 290 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:36:26.368005 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce53f20-cdb1-4066-a6f4-f7978efa2679 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:26.379091 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce53f20-cdb1-4066-a6f4-f7978efa2679 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:26.391541 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a851483-1b87-4db9-bfdf-cf819cd18934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:26.392177 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a851483-1b87-4db9-bfdf-cf819cd18934 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:26.392458 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a851483-1b87-4db9-bfdf-cf819cd18934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:26.392733 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a851483-1b87-4db9-bfdf-cf819cd18934 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:26.393399 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a851483-1b87-4db9-bfdf-cf819cd18934 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:26.395045 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a851483-1b87-4db9-bfdf-cf819cd18934 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:26.406485 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1252/2504] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:36:26.410942 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d4baf-6462-420a-85dd-b1a0f33d5468 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:26.421654 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d4baf-6462-420a-85dd-b1a0f33d5468 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:26.426749 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58522cb1-86d4-4913-b02f-0d8a349ea066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:26.427397 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58522cb1-86d4-4913-b02f-0d8a349ea066 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:26.427678 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58522cb1-86d4-4913-b02f-0d8a349ea066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:26.427946 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58522cb1-86d4-4913-b02f-0d8a349ea066 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:26.428642 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58522cb1-86d4-4913-b02f-0d8a349ea066 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:26.430236 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58522cb1-86d4-4913-b02f-0d8a349ea066 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:26.441359 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1253/2505] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:36:26.445510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a851483-1b87-4db9-bfdf-cf819cd18934 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:26.456295 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a851483-1b87-4db9-bfdf-cf819cd18934 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:26.461432 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92d9573-e19a-4d3d-b080-c31708c9a285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:26.462127 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92d9573-e19a-4d3d-b080-c31708c9a285 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:26.462408 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92d9573-e19a-4d3d-b080-c31708c9a285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:26.462685 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92d9573-e19a-4d3d-b080-c31708c9a285 admin admin] PATH_INFO: `/v3/projects/0fe73d543bb34adb92c4c7f9d0f3c37d/users/0d4baf5e4a7844ee9ca2d7ce96c9a70a/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:26.480144 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92d9573-e19a-4d3d-b080-c31708c9a285 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fe73d543bb34adb92c4c7f9d0f3c37d, user_id=0d4baf5e4a7844ee9ca2d7ce96c9a70a, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:26.483182 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92d9573-e19a-4d3d-b080-c31708c9a285 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:26.501586 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1253/2506] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:36:26 2022] PUT /identity/v3/projects/0fe73d543bb34adb92c4c7f9d0f3c37d/users/0d4baf5e4a7844ee9ca2d7ce96c9a70a/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:26.513323 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08289f1-0712-44c0-8772-5ef39ff4ccf8 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:26.513616 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08289f1-0712-44c0-8772-5ef39ff4ccf8 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:26.513884 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08289f1-0712-44c0-8772-5ef39ff4ccf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:26.544144 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b08289f1-0712-44c0-8772-5ef39ff4ccf8 None None] MFA Rules not processed for user `0d4baf5e4a7844ee9ca2d7ce96c9a70a`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:26.545382 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b08289f1-0712-44c0-8772-5ef39ff4ccf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:26.614177 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1254/2507] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:26.619108 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92d9573-e19a-4d3d-b080-c31708c9a285 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:26.686020 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92d9573-e19a-4d3d-b080-c31708c9a285 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:26.690973 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2969a4f8-dcd4-4be2-a553-fde54c2066c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:26.691616 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2969a4f8-dcd4-4be2-a553-fde54c2066c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:26.691898 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2969a4f8-dcd4-4be2-a553-fde54c2066c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:26.692163 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2969a4f8-dcd4-4be2-a553-fde54c2066c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:26.692830 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2969a4f8-dcd4-4be2-a553-fde54c2066c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:26.696048 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2969a4f8-dcd4-4be2-a553-fde54c2066c7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:26.728003 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1254/2508] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:36:26 2022] POST /identity/v3/projects => generated 381 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:36:26.734585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08289f1-0712-44c0-8772-5ef39ff4ccf8 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:26.748157 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08289f1-0712-44c0-8772-5ef39ff4ccf8 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:26.809843 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a05b9b-c4b6-4a2f-afd9-8c3dfbb5eef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:26.813112 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a05b9b-c4b6-4a2f-afd9-8c3dfbb5eef6 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:26.813112 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a05b9b-c4b6-4a2f-afd9-8c3dfbb5eef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:26.813112 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a05b9b-c4b6-4a2f-afd9-8c3dfbb5eef6 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:26.813112 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a05b9b-c4b6-4a2f-afd9-8c3dfbb5eef6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:26.815974 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a05b9b-c4b6-4a2f-afd9-8c3dfbb5eef6 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:26.842119 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1255/2509] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:36:26 2022] POST /identity/v3/users => generated 297 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:36:26.843399 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2969a4f8-dcd4-4be2-a553-fde54c2066c7 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:26.853641 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2969a4f8-dcd4-4be2-a553-fde54c2066c7 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:26.858843 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da818bfa-92f6-4e64-af3f-203181a617e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:26.859497 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da818bfa-92f6-4e64-af3f-203181a617e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:26.859778 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da818bfa-92f6-4e64-af3f-203181a617e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:26.860050 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da818bfa-92f6-4e64-af3f-203181a617e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:26.860755 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da818bfa-92f6-4e64-af3f-203181a617e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:26.862378 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da818bfa-92f6-4e64-af3f-203181a617e9 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:26.876746 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1255/2510] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:36:26.877948 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a05b9b-c4b6-4a2f-afd9-8c3dfbb5eef6 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:26.888565 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a05b9b-c4b6-4a2f-afd9-8c3dfbb5eef6 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:26.893583 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e2e12f-d16c-40b5-a850-812b43c3e355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:26.894261 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e2e12f-d16c-40b5-a850-812b43c3e355 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:26.894546 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e2e12f-d16c-40b5-a850-812b43c3e355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:26.894852 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e2e12f-d16c-40b5-a850-812b43c3e355 admin admin] PATH_INFO: `/v3/projects/c3f442ee15054919a71532c1496de69f/users/feb5eb6539fb4d47a1db3a7a266dba67/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:26.911753 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e2e12f-d16c-40b5-a850-812b43c3e355 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3f442ee15054919a71532c1496de69f, user_id=feb5eb6539fb4d47a1db3a7a266dba67, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:26.914715 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e2e12f-d16c-40b5-a850-812b43c3e355 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:26.929062 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1256/2511] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:36:26 2022] PUT /identity/v3/projects/c3f442ee15054919a71532c1496de69f/users/feb5eb6539fb4d47a1db3a7a266dba67/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:26.933078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da818bfa-92f6-4e64-af3f-203181a617e9 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:26.943632 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da818bfa-92f6-4e64-af3f-203181a617e9 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:26.976148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:27.003825 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a63d76-d77e-458f-8188-a6e99d94d1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:27.004378 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a63d76-d77e-458f-8188-a6e99d94d1b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:27.004656 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a63d76-d77e-458f-8188-a6e99d94d1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:27.004941 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a63d76-d77e-458f-8188-a6e99d94d1b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:27.005721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a63d76-d77e-458f-8188-a6e99d94d1b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:27.007508 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a63d76-d77e-458f-8188-a6e99d94d1b4 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:27.017859 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1256/2512] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:36:27.022161 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e2e12f-d16c-40b5-a850-812b43c3e355 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:27.033853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e2e12f-d16c-40b5-a850-812b43c3e355 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:27.039096 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1908ac8-6ca5-449c-837a-87190c3c6619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:27.039774 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1908ac8-6ca5-449c-837a-87190c3c6619 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:27.040059 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1908ac8-6ca5-449c-837a-87190c3c6619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:27.040337 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1908ac8-6ca5-449c-837a-87190c3c6619 admin admin] PATH_INFO: `/v3/projects/c3f442ee15054919a71532c1496de69f/users/feb5eb6539fb4d47a1db3a7a266dba67/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:27.042439 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1908ac8-6ca5-449c-837a-87190c3c6619 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3f442ee15054919a71532c1496de69f, user_id=feb5eb6539fb4d47a1db3a7a266dba67, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:27.045231 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1908ac8-6ca5-449c-837a-87190c3c6619 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:27.056083 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1257/2513] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:36:27 2022] PUT /identity/v3/projects/c3f442ee15054919a71532c1496de69f/users/feb5eb6539fb4d47a1db3a7a266dba67/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:27.066840 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39616e8-5702-402a-9569-aaaafb19308a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:27.067147 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39616e8-5702-402a-9569-aaaafb19308a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:27.067419 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39616e8-5702-402a-9569-aaaafb19308a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:27.097010 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d39616e8-5702-402a-9569-aaaafb19308a None None] MFA Rules not processed for user `feb5eb6539fb4d47a1db3a7a266dba67`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:27.098297 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d39616e8-5702-402a-9569-aaaafb19308a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:27.170270 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1257/2514] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:27 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:27.183250 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1908ac8-6ca5-449c-837a-87190c3c6619 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:27.185276 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f1908ac8-6ca5-449c-837a-87190c3c6619 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:27.214446 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:27.247017 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1908ac8-6ca5-449c-837a-87190c3c6619 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:27.251205 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3300c2e-1526-43e7-8101-b4c6f40a509e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:27.251825 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3300c2e-1526-43e7-8101-b4c6f40a509e service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:27.252127 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3300c2e-1526-43e7-8101-b4c6f40a509e service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:27.252399 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3300c2e-1526-43e7-8101-b4c6f40a509e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:27.262288 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3300c2e-1526-43e7-8101-b4c6f40a509e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:27.263888 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3300c2e-1526-43e7-8101-b4c6f40a509e service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:27.273862 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1258/2515] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:36:27 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:36:27.636213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39616e8-5702-402a-9569-aaaafb19308a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:27.646313 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39616e8-5702-402a-9569-aaaafb19308a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:27.651694 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4a6194-f286-4053-9a94-e2edd93c1c55 tempest-VolumeQuotaClassesTest-1470874231 tempest-VolumeQuotaClassesTest-1470874231-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb5eb6539fb4d47a1db3a7a266dba67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3f442ee15054919a71532c1496de69f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:27.652385 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4a6194-f286-4053-9a94-e2edd93c1c55 tempest-VolumeQuotaClassesTest-1470874231 tempest-VolumeQuotaClassesTest-1470874231-project-admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:27.652830 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4a6194-f286-4053-9a94-e2edd93c1c55 tempest-VolumeQuotaClassesTest-1470874231 tempest-VolumeQuotaClassesTest-1470874231-project-admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:27.653111 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4a6194-f286-4053-9a94-e2edd93c1c55 tempest-VolumeQuotaClassesTest-1470874231 tempest-VolumeQuotaClassesTest-1470874231-project-admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:27.653813 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4a6194-f286-4053-9a94-e2edd93c1c55 tempest-VolumeQuotaClassesTest-1470874231 tempest-VolumeQuotaClassesTest-1470874231-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:36:27.654118 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4a6194-f286-4053-9a94-e2edd93c1c55 tempest-VolumeQuotaClassesTest-1470874231 tempest-VolumeQuotaClassesTest-1470874231-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:27.655841 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4a6194-f286-4053-9a94-e2edd93c1c55 tempest-VolumeQuotaClassesTest-1470874231 tempest-VolumeQuotaClassesTest-1470874231-project-admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:27.668412 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1258/2516] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:36:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:36:27.673292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3300c2e-1526-43e7-8101-b4c6f40a509e service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:27.685482 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3300c2e-1526-43e7-8101-b4c6f40a509e service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:27.691505 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aad8522-570a-4162-bdf6-62bd7e3f8dd0 tempest-VolumeQuotaClassesTest-1470874231 tempest-VolumeQuotaClassesTest-1470874231-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb5eb6539fb4d47a1db3a7a266dba67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3f442ee15054919a71532c1496de69f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:27.692147 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad8522-570a-4162-bdf6-62bd7e3f8dd0 tempest-VolumeQuotaClassesTest-1470874231 tempest-VolumeQuotaClassesTest-1470874231-project-admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:27.692428 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad8522-570a-4162-bdf6-62bd7e3f8dd0 tempest-VolumeQuotaClassesTest-1470874231 tempest-VolumeQuotaClassesTest-1470874231-project-admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:27.692698 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad8522-570a-4162-bdf6-62bd7e3f8dd0 tempest-VolumeQuotaClassesTest-1470874231 tempest-VolumeQuotaClassesTest-1470874231-project-admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:27.693363 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aad8522-570a-4162-bdf6-62bd7e3f8dd0 tempest-VolumeQuotaClassesTest-1470874231 tempest-VolumeQuotaClassesTest-1470874231-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:27.695333 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aad8522-570a-4162-bdf6-62bd7e3f8dd0 tempest-VolumeQuotaClassesTest-1470874231 tempest-VolumeQuotaClassesTest-1470874231-project-admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:27.727079 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1259/2517] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:36:27 2022] POST /identity/v3/projects => generated 355 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:36:27.731966 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4a6194-f286-4053-9a94-e2edd93c1c55 tempest-VolumeQuotaClassesTest-1470874231 tempest-VolumeQuotaClassesTest-1470874231-project-admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:27.746732 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4a6194-f286-4053-9a94-e2edd93c1c55 tempest-VolumeQuotaClassesTest-1470874231 tempest-VolumeQuotaClassesTest-1470874231-project-admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:27.807687 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f0fa8-0a55-4033-82f4-c40c4d5072f8 tempest-VolumeQuotaClassesTest-1470874231 tempest-VolumeQuotaClassesTest-1470874231-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb5eb6539fb4d47a1db3a7a266dba67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3f442ee15054919a71532c1496de69f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:27.808205 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f0fa8-0a55-4033-82f4-c40c4d5072f8 tempest-VolumeQuotaClassesTest-1470874231 tempest-VolumeQuotaClassesTest-1470874231-project-admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:27.808729 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f0fa8-0a55-4033-82f4-c40c4d5072f8 tempest-VolumeQuotaClassesTest-1470874231 tempest-VolumeQuotaClassesTest-1470874231-project-admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:27.809180 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f0fa8-0a55-4033-82f4-c40c4d5072f8 tempest-VolumeQuotaClassesTest-1470874231 tempest-VolumeQuotaClassesTest-1470874231-project-admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:27.809877 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547f0fa8-0a55-4033-82f4-c40c4d5072f8 tempest-VolumeQuotaClassesTest-1470874231 tempest-VolumeQuotaClassesTest-1470874231-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:36:27.810214 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547f0fa8-0a55-4033-82f4-c40c4d5072f8 tempest-VolumeQuotaClassesTest-1470874231 tempest-VolumeQuotaClassesTest-1470874231-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:27.811840 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547f0fa8-0a55-4033-82f4-c40c4d5072f8 tempest-VolumeQuotaClassesTest-1470874231 tempest-VolumeQuotaClassesTest-1470874231-project-admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:27.824879 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1259/2518] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:36:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:36:27.866094 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aad8522-570a-4162-bdf6-62bd7e3f8dd0 tempest-VolumeQuotaClassesTest-1470874231 tempest-VolumeQuotaClassesTest-1470874231-project-admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:27.877078 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aad8522-570a-4162-bdf6-62bd7e3f8dd0 tempest-VolumeQuotaClassesTest-1470874231 tempest-VolumeQuotaClassesTest-1470874231-project-admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:27.881907 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b8e84-493a-487d-867a-dedc694484e0 tempest-VolumeQuotaClassesTest-1470874231 tempest-VolumeQuotaClassesTest-1470874231-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb5eb6539fb4d47a1db3a7a266dba67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3f442ee15054919a71532c1496de69f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:27.882546 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b8e84-493a-487d-867a-dedc694484e0 tempest-VolumeQuotaClassesTest-1470874231 tempest-VolumeQuotaClassesTest-1470874231-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:27.882830 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b8e84-493a-487d-867a-dedc694484e0 tempest-VolumeQuotaClassesTest-1470874231 tempest-VolumeQuotaClassesTest-1470874231-project-admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:27.883134 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b8e84-493a-487d-867a-dedc694484e0 tempest-VolumeQuotaClassesTest-1470874231 tempest-VolumeQuotaClassesTest-1470874231-project-admin] PATH_INFO: `/v3/projects/0920510cd3364e9e90e832d72d1126ac` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:27.884319 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4b8e84-493a-487d-867a-dedc694484e0 tempest-VolumeQuotaClassesTest-1470874231 tempest-VolumeQuotaClassesTest-1470874231-project-admin] RBAC: Authorizing `identity:delete_project(project_id=0920510cd3364e9e90e832d72d1126ac)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:27.886231 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4b8e84-493a-487d-867a-dedc694484e0 tempest-VolumeQuotaClassesTest-1470874231 tempest-VolumeQuotaClassesTest-1470874231-project-admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:27.936404 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0e4b8e84-493a-487d-867a-dedc694484e0 tempest-VolumeQuotaClassesTest-1470874231 tempest-VolumeQuotaClassesTest-1470874231-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0920510cd3364e9e90e832d72d1126ac', 'request_id': 'req-0e4b8e84-493a-487d-867a-dedc694484e0'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:27.942263 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0e4b8e84-493a-487d-867a-dedc694484e0 tempest-VolumeQuotaClassesTest-1470874231 tempest-VolumeQuotaClassesTest-1470874231-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0920510cd3364e9e90e832d72d1126ac', 'request_id': 'req-0e4b8e84-493a-487d-867a-dedc694484e0'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:27.955871 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0e4b8e84-493a-487d-867a-dedc694484e0 tempest-VolumeQuotaClassesTest-1470874231 tempest-VolumeQuotaClassesTest-1470874231-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0920510cd3364e9e90e832d72d1126ac', 'request_id': 'req-0e4b8e84-493a-487d-867a-dedc694484e0'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:27.968950 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0e4b8e84-493a-487d-867a-dedc694484e0 tempest-VolumeQuotaClassesTest-1470874231 tempest-VolumeQuotaClassesTest-1470874231-project-admin] The token cache is being invalidate because project 0920510cd3364e9e90e832d72d1126ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:36:27.969235 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0e4b8e84-493a-487d-867a-dedc694484e0 tempest-VolumeQuotaClassesTest-1470874231 tempest-VolumeQuotaClassesTest-1470874231-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:27.969889 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0e4b8e84-493a-487d-867a-dedc694484e0 tempest-VolumeQuotaClassesTest-1470874231 tempest-VolumeQuotaClassesTest-1470874231-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:27.971481 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1260/2519] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:36:27 2022] DELETE /identity/v3/projects/0920510cd3364e9e90e832d72d1126ac => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:28.032680 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f0fa8-0a55-4033-82f4-c40c4d5072f8 tempest-VolumeQuotaClassesTest-1470874231 tempest-VolumeQuotaClassesTest-1470874231-project-admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:28.034804 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-547f0fa8-0a55-4033-82f4-c40c4d5072f8 tempest-VolumeQuotaClassesTest-1470874231 tempest-VolumeQuotaClassesTest-1470874231-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:28.080875 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:28.101381 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f0fa8-0a55-4033-82f4-c40c4d5072f8 tempest-VolumeQuotaClassesTest-1470874231 tempest-VolumeQuotaClassesTest-1470874231-project-admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:28.156795 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e815385c-bb50-42ae-b9f0-33495daaf723 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:28.157428 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815385c-bb50-42ae-b9f0-33495daaf723 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:28.157706 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815385c-bb50-42ae-b9f0-33495daaf723 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:28.158005 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815385c-bb50-42ae-b9f0-33495daaf723 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:28.160753 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e815385c-bb50-42ae-b9f0-33495daaf723 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:28.199774 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b8e84-493a-487d-867a-dedc694484e0 tempest-VolumeQuotaClassesTest-1470874231 tempest-VolumeQuotaClassesTest-1470874231-project-admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:28.201055 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0e4b8e84-493a-487d-867a-dedc694484e0 tempest-VolumeQuotaClassesTest-1470874231 tempest-VolumeQuotaClassesTest-1470874231-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:28.232718 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e815385c-bb50-42ae-b9f0-33495daaf723 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:28.234238 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e815385c-bb50-42ae-b9f0-33495daaf723 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:28.244334 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1260/2520] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:36:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:36:28.267869 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b8e84-493a-487d-867a-dedc694484e0 tempest-VolumeQuotaClassesTest-1470874231 tempest-VolumeQuotaClassesTest-1470874231-project-admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:28.272303 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b10f9-d18e-4c99-b980-56b83c698589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:28.272950 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b10f9-d18e-4c99-b980-56b83c698589 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:28.273268 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b10f9-d18e-4c99-b980-56b83c698589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:28.273526 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b10f9-d18e-4c99-b980-56b83c698589 admin admin] PATH_INFO: `/v3/users/0d4baf5e4a7844ee9ca2d7ce96c9a70a` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:28.274731 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4b10f9-d18e-4c99-b980-56b83c698589 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d4baf5e4a7844ee9ca2d7ce96c9a70a)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:28.276675 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4b10f9-d18e-4c99-b980-56b83c698589 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:28.447098 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:28.464193 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ec4b10f9-d18e-4c99-b980-56b83c698589 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d4baf5e4a7844ee9ca2d7ce96c9a70a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:28.473569 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ec4b10f9-d18e-4c99-b980-56b83c698589 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d4baf5e4a7844ee9ca2d7ce96c9a70a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:28.474273 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ec4b10f9-d18e-4c99-b980-56b83c698589 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d4baf5e4a7844ee9ca2d7ce96c9a70a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:28.489115 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ec4b10f9-d18e-4c99-b980-56b83c698589 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d4baf5e4a7844ee9ca2d7ce96c9a70a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:28.489972 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ec4b10f9-d18e-4c99-b980-56b83c698589 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d4baf5e4a7844ee9ca2d7ce96c9a70a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:28.504140 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1261/2521] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:36:28 2022] DELETE /identity/v3/users/0d4baf5e4a7844ee9ca2d7ce96c9a70a => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:28.508800 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db12d5-ff5e-49d5-9036-2409a6116af3 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:28.509090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db12d5-ff5e-49d5-9036-2409a6116af3 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:28.509366 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db12d5-ff5e-49d5-9036-2409a6116af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:28.530560 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a3db12d5-ff5e-49d5-9036-2409a6116af3 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:28.531827 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a3db12d5-ff5e-49d5-9036-2409a6116af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:28.599879 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1261/2522] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:28.610111 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b10f9-d18e-4c99-b980-56b83c698589 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:28.611712 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ec4b10f9-d18e-4c99-b980-56b83c698589 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:28.670564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b10f9-d18e-4c99-b980-56b83c698589 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:28.674524 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaef392-a52d-4e8d-bc4a-1f1faf3a28c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:28.675178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaef392-a52d-4e8d-bc4a-1f1faf3a28c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:28.675419 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaef392-a52d-4e8d-bc4a-1f1faf3a28c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:28.675708 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaef392-a52d-4e8d-bc4a-1f1faf3a28c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:28.685614 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaef392-a52d-4e8d-bc4a-1f1faf3a28c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:28.687124 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaef392-a52d-4e8d-bc4a-1f1faf3a28c8 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:28.696945 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1262/2523] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:36:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:36:28.971967 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db12d5-ff5e-49d5-9036-2409a6116af3 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:28.974125 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a3db12d5-ff5e-49d5-9036-2409a6116af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:28.988537 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db12d5-ff5e-49d5-9036-2409a6116af3 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:28.994502 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf421562-8695-44ae-89b8-ee1c08c97d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:28.995206 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf421562-8695-44ae-89b8-ee1c08c97d15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:28.995514 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf421562-8695-44ae-89b8-ee1c08c97d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:28.995758 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf421562-8695-44ae-89b8-ee1c08c97d15 admin admin] PATH_INFO: `/v3/projects/0fe73d543bb34adb92c4c7f9d0f3c37d` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:28.997749 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf421562-8695-44ae-89b8-ee1c08c97d15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fe73d543bb34adb92c4c7f9d0f3c37d)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:29.000481 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf421562-8695-44ae-89b8-ee1c08c97d15 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:29.045487 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:29.052997 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-bf421562-8695-44ae-89b8-ee1c08c97d15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fe73d543bb34adb92c4c7f9d0f3c37d', 'request_id': 'req-bf421562-8695-44ae-89b8-ee1c08c97d15'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:29.062418 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-bf421562-8695-44ae-89b8-ee1c08c97d15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fe73d543bb34adb92c4c7f9d0f3c37d', 'request_id': 'req-bf421562-8695-44ae-89b8-ee1c08c97d15'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:29.068153 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-bf421562-8695-44ae-89b8-ee1c08c97d15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fe73d543bb34adb92c4c7f9d0f3c37d', 'request_id': 'req-bf421562-8695-44ae-89b8-ee1c08c97d15'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:29.079438 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-bf421562-8695-44ae-89b8-ee1c08c97d15 admin admin] The token cache is being invalidate because project 0fe73d543bb34adb92c4c7f9d0f3c37d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:36:29.079724 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-bf421562-8695-44ae-89b8-ee1c08c97d15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:29.080358 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-bf421562-8695-44ae-89b8-ee1c08c97d15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:29.082117 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1262/2524] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:36:28 2022] DELETE /identity/v3/projects/0fe73d543bb34adb92c4c7f9d0f3c37d => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:29.085959 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaef392-a52d-4e8d-bc4a-1f1faf3a28c8 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:29.095911 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaef392-a52d-4e8d-bc4a-1f1faf3a28c8 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:29.153459 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe96685-73c2-472b-9dbd-b6d29b2e8441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:29.154119 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe96685-73c2-472b-9dbd-b6d29b2e8441 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:29.154847 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe96685-73c2-472b-9dbd-b6d29b2e8441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:29.154847 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe96685-73c2-472b-9dbd-b6d29b2e8441 admin admin] PATH_INFO: `/v3/users/feb5eb6539fb4d47a1db3a7a266dba67` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:29.155965 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe96685-73c2-472b-9dbd-b6d29b2e8441 admin admin] RBAC: Authorizing `identity:delete_user(user_id=feb5eb6539fb4d47a1db3a7a266dba67)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:29.157894 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe96685-73c2-472b-9dbd-b6d29b2e8441 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:29.209194 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2fe96685-73c2-472b-9dbd-b6d29b2e8441 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'feb5eb6539fb4d47a1db3a7a266dba67'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:29.212864 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:29.218003 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2fe96685-73c2-472b-9dbd-b6d29b2e8441 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'feb5eb6539fb4d47a1db3a7a266dba67'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:29.219507 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2fe96685-73c2-472b-9dbd-b6d29b2e8441 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'feb5eb6539fb4d47a1db3a7a266dba67'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:29.233224 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2fe96685-73c2-472b-9dbd-b6d29b2e8441 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'feb5eb6539fb4d47a1db3a7a266dba67'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:29.234141 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2fe96685-73c2-472b-9dbd-b6d29b2e8441 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'feb5eb6539fb4d47a1db3a7a266dba67'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:29.246201 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1263/2525] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:36:29 2022] DELETE /identity/v3/users/feb5eb6539fb4d47a1db3a7a266dba67 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:29.839461 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf421562-8695-44ae-89b8-ee1c08c97d15 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:29.841317 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bf421562-8695-44ae-89b8-ee1c08c97d15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:29.855071 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf421562-8695-44ae-89b8-ee1c08c97d15 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:29.927149 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b12f03-5eaa-4d07-bd6a-5a6ded0d2b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:29.927806 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b12f03-5eaa-4d07-bd6a-5a6ded0d2b17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:29.928085 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b12f03-5eaa-4d07-bd6a-5a6ded0d2b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:29.928358 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b12f03-5eaa-4d07-bd6a-5a6ded0d2b17 admin admin] PATH_INFO: `/v3/projects/c3f442ee15054919a71532c1496de69f` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:29.929537 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b12f03-5eaa-4d07-bd6a-5a6ded0d2b17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3f442ee15054919a71532c1496de69f)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:29.931490 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b12f03-5eaa-4d07-bd6a-5a6ded0d2b17 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:29.979092 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-21b12f03-5eaa-4d07-bd6a-5a6ded0d2b17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3f442ee15054919a71532c1496de69f', 'request_id': 'req-21b12f03-5eaa-4d07-bd6a-5a6ded0d2b17'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:29.988134 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-21b12f03-5eaa-4d07-bd6a-5a6ded0d2b17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3f442ee15054919a71532c1496de69f', 'request_id': 'req-21b12f03-5eaa-4d07-bd6a-5a6ded0d2b17'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:29.995446 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-21b12f03-5eaa-4d07-bd6a-5a6ded0d2b17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3f442ee15054919a71532c1496de69f', 'request_id': 'req-21b12f03-5eaa-4d07-bd6a-5a6ded0d2b17'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:30.007407 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-21b12f03-5eaa-4d07-bd6a-5a6ded0d2b17 admin admin] The token cache is being invalidate because project c3f442ee15054919a71532c1496de69f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:36:30.007692 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-21b12f03-5eaa-4d07-bd6a-5a6ded0d2b17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:30.008365 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-21b12f03-5eaa-4d07-bd6a-5a6ded0d2b17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:30.009892 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1263/2526] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:36:29 2022] DELETE /identity/v3/projects/c3f442ee15054919a71532c1496de69f => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:30.015612 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee42b1-ab9c-43c5-8338-79d933e84a23 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:30.016143 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee42b1-ab9c-43c5-8338-79d933e84a23 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:30.016349 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee42b1-ab9c-43c5-8338-79d933e84a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:30.039445 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-99ee42b1-ab9c-43c5-8338-79d933e84a23 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:30.040666 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-99ee42b1-ab9c-43c5-8338-79d933e84a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:30.109330 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1264/2527] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:30.117246 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcea077-23bd-4b7c-9ca2-eb422f391239 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:30.117530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcea077-23bd-4b7c-9ca2-eb422f391239 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:30.117805 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcea077-23bd-4b7c-9ca2-eb422f391239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:30.140158 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-dbcea077-23bd-4b7c-9ca2-eb422f391239 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:30.141396 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-dbcea077-23bd-4b7c-9ca2-eb422f391239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:30.172227 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1264/2528] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:30.176163 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ee42b1-ab9c-43c5-8338-79d933e84a23 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:30.187719 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ee42b1-ab9c-43c5-8338-79d933e84a23 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:30.193798 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaca931-fc7d-4121-b48c-6c8984a12967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:30.194527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaca931-fc7d-4121-b48c-6c8984a12967 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:30.194807 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaca931-fc7d-4121-b48c-6c8984a12967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:30.195104 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaca931-fc7d-4121-b48c-6c8984a12967 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:30.195807 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaca931-fc7d-4121-b48c-6c8984a12967 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:36:30.196145 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaca931-fc7d-4121-b48c-6c8984a12967 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:30.197767 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaca931-fc7d-4121-b48c-6c8984a12967 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:30.210381 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1265/2529] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:36:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:36:30.214802 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcea077-23bd-4b7c-9ca2-eb422f391239 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:30.224953 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcea077-23bd-4b7c-9ca2-eb422f391239 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:30.232226 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a67c58f-4b0b-484a-be97-53a676095625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:30.232863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67c58f-4b0b-484a-be97-53a676095625 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:30.233149 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67c58f-4b0b-484a-be97-53a676095625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:30.233422 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67c58f-4b0b-484a-be97-53a676095625 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:30.234378 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a67c58f-4b0b-484a-be97-53a676095625 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:30.236376 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a67c58f-4b0b-484a-be97-53a676095625 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:30.270753 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1265/2530] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:36:30 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:36:30.275037 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaca931-fc7d-4121-b48c-6c8984a12967 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:30.285439 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaca931-fc7d-4121-b48c-6c8984a12967 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:30.353850 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742b46f0-a37b-42a7-8d10-2d770a73d017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:30.355282 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b46f0-a37b-42a7-8d10-2d770a73d017 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:30.355282 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b46f0-a37b-42a7-8d10-2d770a73d017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:30.355282 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b46f0-a37b-42a7-8d10-2d770a73d017 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:30.355960 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742b46f0-a37b-42a7-8d10-2d770a73d017 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:30.359026 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742b46f0-a37b-42a7-8d10-2d770a73d017 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:30.387551 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1266/2531] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:36:30 2022] POST /identity/v3/users => generated 290 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:36:30.393019 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a67c58f-4b0b-484a-be97-53a676095625 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:30.405995 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a67c58f-4b0b-484a-be97-53a676095625 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:30.413937 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5296548b-de21-44b9-83d9-1bbbd7573b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:30.414166 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5296548b-de21-44b9-83d9-1bbbd7573b31 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:30.414336 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5296548b-de21-44b9-83d9-1bbbd7573b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:30.414611 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5296548b-de21-44b9-83d9-1bbbd7573b31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:30.415959 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5296548b-de21-44b9-83d9-1bbbd7573b31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:30.418760 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5296548b-de21-44b9-83d9-1bbbd7573b31 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:30.433360 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1266/2532] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:36:30 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:36:30.438008 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742b46f0-a37b-42a7-8d10-2d770a73d017 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:30.451082 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742b46f0-a37b-42a7-8d10-2d770a73d017 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:30.458380 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e381d3c-be50-475f-af1b-a69b9b082ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:30.459232 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e381d3c-be50-475f-af1b-a69b9b082ee9 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:30.459514 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e381d3c-be50-475f-af1b-a69b9b082ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:30.459789 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e381d3c-be50-475f-af1b-a69b9b082ee9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:30.461062 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e381d3c-be50-475f-af1b-a69b9b082ee9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:30.463458 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e381d3c-be50-475f-af1b-a69b9b082ee9 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:30.479323 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1267/2533] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:36:30 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:36:30.480645 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5296548b-de21-44b9-83d9-1bbbd7573b31 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:30.494443 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5296548b-de21-44b9-83d9-1bbbd7573b31 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:30.501210 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc859f2-daa7-4d7a-a146-f129cdac30e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:30.502583 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc859f2-daa7-4d7a-a146-f129cdac30e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:30.502872 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc859f2-daa7-4d7a-a146-f129cdac30e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:30.503185 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc859f2-daa7-4d7a-a146-f129cdac30e3 admin admin] PATH_INFO: `/v3/projects/9304c454ab314b68a1c0a88eee1d418c/users/fa47803ab05a4b16bdfcdee85b1cdb9d/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:30.523226 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc859f2-daa7-4d7a-a146-f129cdac30e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9304c454ab314b68a1c0a88eee1d418c, user_id=fa47803ab05a4b16bdfcdee85b1cdb9d, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:30.525838 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc859f2-daa7-4d7a-a146-f129cdac30e3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:30.529526 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:30.551006 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1267/2534] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:36:30 2022] PUT /identity/v3/projects/9304c454ab314b68a1c0a88eee1d418c/users/fa47803ab05a4b16bdfcdee85b1cdb9d/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:30.565914 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d87f2be-9431-45b6-94b2-be6b55a5afa7 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:30.566389 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d87f2be-9431-45b6-94b2-be6b55a5afa7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:30.566737 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d87f2be-9431-45b6-94b2-be6b55a5afa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:30.603716 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-2d87f2be-9431-45b6-94b2-be6b55a5afa7 None None] MFA Rules not processed for user `fa47803ab05a4b16bdfcdee85b1cdb9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:30.605952 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2d87f2be-9431-45b6-94b2-be6b55a5afa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:30.635925 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:30.699314 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1268/2535] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:30.704066 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc859f2-daa7-4d7a-a146-f129cdac30e3 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:30.797169 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc859f2-daa7-4d7a-a146-f129cdac30e3 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:30.802359 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266cfeb0-66b6-44ea-94e7-611ef6d49cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:30.803121 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266cfeb0-66b6-44ea-94e7-611ef6d49cec admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:30.803364 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266cfeb0-66b6-44ea-94e7-611ef6d49cec admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:30.803655 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266cfeb0-66b6-44ea-94e7-611ef6d49cec admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:30.804441 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266cfeb0-66b6-44ea-94e7-611ef6d49cec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:30.806537 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266cfeb0-66b6-44ea-94e7-611ef6d49cec admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:30.839036 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1268/2536] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:36:30 2022] POST /identity/v3/projects => generated 381 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:36:30.844952 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d87f2be-9431-45b6-94b2-be6b55a5afa7 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:30.857115 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d87f2be-9431-45b6-94b2-be6b55a5afa7 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:30.931322 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491bcfe1-fb48-472f-94d1-b131e36ee6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:30.932040 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491bcfe1-fb48-472f-94d1-b131e36ee6fa admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:30.932348 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491bcfe1-fb48-472f-94d1-b131e36ee6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:30.932597 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491bcfe1-fb48-472f-94d1-b131e36ee6fa admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:30.933408 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491bcfe1-fb48-472f-94d1-b131e36ee6fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:30.935957 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491bcfe1-fb48-472f-94d1-b131e36ee6fa admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:30.961246 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1269/2537] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:36:30 2022] POST /identity/v3/users => generated 297 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:36:30.965342 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266cfeb0-66b6-44ea-94e7-611ef6d49cec admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:30.980969 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266cfeb0-66b6-44ea-94e7-611ef6d49cec admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:30.987864 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b47a84-8379-4da7-b897-527e16ac9d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:30.988662 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b47a84-8379-4da7-b897-527e16ac9d70 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:30.988947 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b47a84-8379-4da7-b897-527e16ac9d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:30.989218 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b47a84-8379-4da7-b897-527e16ac9d70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:30.989919 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b47a84-8379-4da7-b897-527e16ac9d70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:30.991633 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b47a84-8379-4da7-b897-527e16ac9d70 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:31.007467 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1269/2538] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:36:30 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:36:31.008686 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491bcfe1-fb48-472f-94d1-b131e36ee6fa admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:31.020066 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491bcfe1-fb48-472f-94d1-b131e36ee6fa admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:31.025757 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec3466-f2ed-4673-8d79-187330b5cb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:31.026524 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec3466-f2ed-4673-8d79-187330b5cb99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:31.026744 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec3466-f2ed-4673-8d79-187330b5cb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:31.027472 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec3466-f2ed-4673-8d79-187330b5cb99 admin admin] PATH_INFO: `/v3/projects/7699ea2d983e40e986d641915a4fee71/users/9b643a52d15c4ee39ddc20f7ea8f0ba5/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:31.046078 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfec3466-f2ed-4673-8d79-187330b5cb99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7699ea2d983e40e986d641915a4fee71, user_id=9b643a52d15c4ee39ddc20f7ea8f0ba5, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:31.049061 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfec3466-f2ed-4673-8d79-187330b5cb99 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:31.065180 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1270/2539] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:36:31 2022] PUT /identity/v3/projects/7699ea2d983e40e986d641915a4fee71/users/9b643a52d15c4ee39ddc20f7ea8f0ba5/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:36:31.070169 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b47a84-8379-4da7-b897-527e16ac9d70 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:31.086742 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b47a84-8379-4da7-b897-527e16ac9d70 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:31.181285 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c21486-4c93-480c-8f40-e9854a17c217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:31.181938 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c21486-4c93-480c-8f40-e9854a17c217 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:31.182219 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c21486-4c93-480c-8f40-e9854a17c217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:31.182489 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c21486-4c93-480c-8f40-e9854a17c217 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:31.183654 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c21486-4c93-480c-8f40-e9854a17c217 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:31.185900 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c21486-4c93-480c-8f40-e9854a17c217 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:31.203010 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1270/2540] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:36:31 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 10:36:31.204444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec3466-f2ed-4673-8d79-187330b5cb99 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:31.220696 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec3466-f2ed-4673-8d79-187330b5cb99 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:31.225264 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c348c-6b37-413d-8cea-b31e0514616f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:31.226585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c348c-6b37-413d-8cea-b31e0514616f admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:31.227065 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c348c-6b37-413d-8cea-b31e0514616f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:31.227444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c348c-6b37-413d-8cea-b31e0514616f admin admin] PATH_INFO: `/v3/projects/7699ea2d983e40e986d641915a4fee71/users/9b643a52d15c4ee39ddc20f7ea8f0ba5/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:31.229678 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313c348c-6b37-413d-8cea-b31e0514616f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7699ea2d983e40e986d641915a4fee71, user_id=9b643a52d15c4ee39ddc20f7ea8f0ba5, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:31.233915 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313c348c-6b37-413d-8cea-b31e0514616f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:31.250991 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1271/2541] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:36:31 2022] PUT /identity/v3/projects/7699ea2d983e40e986d641915a4fee71/users/9b643a52d15c4ee39ddc20f7ea8f0ba5/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:31.258372 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b628e7-985c-47fa-80f2-25cfc66442e2 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:31.258674 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b628e7-985c-47fa-80f2-25cfc66442e2 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:31.258963 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b628e7-985c-47fa-80f2-25cfc66442e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:31.300199 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-08b628e7-985c-47fa-80f2-25cfc66442e2 None None] MFA Rules not processed for user `9b643a52d15c4ee39ddc20f7ea8f0ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:31.301458 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-08b628e7-985c-47fa-80f2-25cfc66442e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:31.362241 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:31.393522 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1271/2542] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:31 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:31.406194 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c348c-6b37-413d-8cea-b31e0514616f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:31.407949 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-313c348c-6b37-413d-8cea-b31e0514616f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:31.486556 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c348c-6b37-413d-8cea-b31e0514616f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:31.490549 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6d0436-034e-494b-b287-ec850327f254 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:31.491348 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d0436-034e-494b-b287-ec850327f254 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:31.491695 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d0436-034e-494b-b287-ec850327f254 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:31.492031 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d0436-034e-494b-b287-ec850327f254 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:31.501997 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6d0436-034e-494b-b287-ec850327f254 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:31.504047 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6d0436-034e-494b-b287-ec850327f254 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:31.546570 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1272/2543] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:36:31 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:36:32.103147 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b628e7-985c-47fa-80f2-25cfc66442e2 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:32.108444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6d0436-034e-494b-b287-ec850327f254 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:32.109858 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-cc6d0436-034e-494b-b287-ec850327f254 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:32.139237 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:32.185216 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b628e7-985c-47fa-80f2-25cfc66442e2 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:32.193133 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb1dc5e-3708-4623-8686-88881dba9d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:32.193503 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6d0436-034e-494b-b287-ec850327f254 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:32.193795 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1dc5e-3708-4623-8686-88881dba9d41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:32.194393 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1dc5e-3708-4623-8686-88881dba9d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:32.194623 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1dc5e-3708-4623-8686-88881dba9d41 admin admin] PATH_INFO: `/v3/users/fa47803ab05a4b16bdfcdee85b1cdb9d` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:32.195830 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb1dc5e-3708-4623-8686-88881dba9d41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa47803ab05a4b16bdfcdee85b1cdb9d)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:32.198373 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4802026c-b6cb-4b23-9ed7-9516e6edea8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:32.198517 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb1dc5e-3708-4623-8686-88881dba9d41 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:32.199312 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4802026c-b6cb-4b23-9ed7-9516e6edea8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:32.199601 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4802026c-b6cb-4b23-9ed7-9516e6edea8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:32.199876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4802026c-b6cb-4b23-9ed7-9516e6edea8f admin admin] PATH_INFO: `/v3/users/706dbfa719c04fd0a3704ebd7eb812cd` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:32.213331 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4802026c-b6cb-4b23-9ed7-9516e6edea8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=706dbfa719c04fd0a3704ebd7eb812cd)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:32.215392 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4802026c-b6cb-4b23-9ed7-9516e6edea8f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:32.252105 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ddb1dc5e-3708-4623-8686-88881dba9d41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa47803ab05a4b16bdfcdee85b1cdb9d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:32.252845 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ddb1dc5e-3708-4623-8686-88881dba9d41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa47803ab05a4b16bdfcdee85b1cdb9d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:32.253540 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ddb1dc5e-3708-4623-8686-88881dba9d41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa47803ab05a4b16bdfcdee85b1cdb9d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:32.262202 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ddb1dc5e-3708-4623-8686-88881dba9d41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa47803ab05a4b16bdfcdee85b1cdb9d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:32.263336 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4802026c-b6cb-4b23-9ed7-9516e6edea8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '706dbfa719c04fd0a3704ebd7eb812cd'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:32.270611 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ddb1dc5e-3708-4623-8686-88881dba9d41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa47803ab05a4b16bdfcdee85b1cdb9d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:32.272112 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4802026c-b6cb-4b23-9ed7-9516e6edea8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '706dbfa719c04fd0a3704ebd7eb812cd'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:32.272838 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4802026c-b6cb-4b23-9ed7-9516e6edea8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '706dbfa719c04fd0a3704ebd7eb812cd'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:32.285776 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4802026c-b6cb-4b23-9ed7-9516e6edea8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '706dbfa719c04fd0a3704ebd7eb812cd'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:32.286439 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4802026c-b6cb-4b23-9ed7-9516e6edea8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '706dbfa719c04fd0a3704ebd7eb812cd'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:32.288363 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1272/2544] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:36:32 2022] DELETE /identity/v3/users/fa47803ab05a4b16bdfcdee85b1cdb9d => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:32.292738 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae45f6-ddd5-47ab-8a22-a1ab16abee78 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:32.293023 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae45f6-ddd5-47ab-8a22-a1ab16abee78 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:32.293322 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae45f6-ddd5-47ab-8a22-a1ab16abee78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:32.299897 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1273/2545] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:36:32 2022] DELETE /identity/v3/users/706dbfa719c04fd0a3704ebd7eb812cd => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:32.312287 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b6ae45f6-ddd5-47ab-8a22-a1ab16abee78 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:32.313503 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b6ae45f6-ddd5-47ab-8a22-a1ab16abee78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:32.379919 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1273/2546] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:36:32.390177 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4802026c-b6cb-4b23-9ed7-9516e6edea8f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:32.391702 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4802026c-b6cb-4b23-9ed7-9516e6edea8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:32.452113 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4802026c-b6cb-4b23-9ed7-9516e6edea8f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:32.455888 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7745be1a-f1cc-4f03-b631-3aa6548aa5d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:32.457506 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7745be1a-f1cc-4f03-b631-3aa6548aa5d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:32.457792 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7745be1a-f1cc-4f03-b631-3aa6548aa5d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:32.458061 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7745be1a-f1cc-4f03-b631-3aa6548aa5d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:32.469695 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7745be1a-f1cc-4f03-b631-3aa6548aa5d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:32.471178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7745be1a-f1cc-4f03-b631-3aa6548aa5d2 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:32.488069 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1274/2547] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:36:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:36:32.489516 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ae45f6-ddd5-47ab-8a22-a1ab16abee78 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:32.491325 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b6ae45f6-ddd5-47ab-8a22-a1ab16abee78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:32.526563 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ae45f6-ddd5-47ab-8a22-a1ab16abee78 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:32.536298 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d4af28-d7a1-483a-b68c-7202d174a349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:32.537752 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4af28-d7a1-483a-b68c-7202d174a349 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:32.538378 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4af28-d7a1-483a-b68c-7202d174a349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:32.538848 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4af28-d7a1-483a-b68c-7202d174a349 admin admin] PATH_INFO: `/v3/projects/588c79a0604e49f49123980f0e9af63a` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:32.542973 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:32.553973 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d4af28-d7a1-483a-b68c-7202d174a349 admin admin] RBAC: Authorizing `identity:delete_project(project_id=588c79a0604e49f49123980f0e9af63a)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:32.556007 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d4af28-d7a1-483a-b68c-7202d174a349 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:32.610888 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e8d4af28-d7a1-483a-b68c-7202d174a349 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '588c79a0604e49f49123980f0e9af63a', 'request_id': 'req-e8d4af28-d7a1-483a-b68c-7202d174a349'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:32.620066 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e8d4af28-d7a1-483a-b68c-7202d174a349 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '588c79a0604e49f49123980f0e9af63a', 'request_id': 'req-e8d4af28-d7a1-483a-b68c-7202d174a349'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:32.626143 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e8d4af28-d7a1-483a-b68c-7202d174a349 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '588c79a0604e49f49123980f0e9af63a', 'request_id': 'req-e8d4af28-d7a1-483a-b68c-7202d174a349'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:32.639279 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e8d4af28-d7a1-483a-b68c-7202d174a349 admin admin] The token cache is being invalidate because project 588c79a0604e49f49123980f0e9af63a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:36:32.639906 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e8d4af28-d7a1-483a-b68c-7202d174a349 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:32.640585 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e8d4af28-d7a1-483a-b68c-7202d174a349 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:32.642330 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1274/2548] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:36:32 2022] DELETE /identity/v3/projects/588c79a0604e49f49123980f0e9af63a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:32.647306 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5720e-fa48-4e0d-a88a-70a6f355a812 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:32.647597 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5720e-fa48-4e0d-a88a-70a6f355a812 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:32.647854 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5720e-fa48-4e0d-a88a-70a6f355a812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:32.668117 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-78d5720e-fa48-4e0d-a88a-70a6f355a812 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:32.669314 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-78d5720e-fa48-4e0d-a88a-70a6f355a812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:32.694582 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:32.731243 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1275/2549] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:32.739149 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc60d4-dd44-4092-9eea-96f6241d650c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:32.739442 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc60d4-dd44-4092-9eea-96f6241d650c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:32.739716 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc60d4-dd44-4092-9eea-96f6241d650c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:32.763697 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9fbc60d4-dd44-4092-9eea-96f6241d650c None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:32.765655 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9fbc60d4-dd44-4092-9eea-96f6241d650c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:32.773014 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d5720e-fa48-4e0d-a88a-70a6f355a812 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:32.774600 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-78d5720e-fa48-4e0d-a88a-70a6f355a812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:32.780745 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1275/2550] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:32.781694 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc60d4-dd44-4092-9eea-96f6241d650c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:32.787900 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d5720e-fa48-4e0d-a88a-70a6f355a812 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:32.792995 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa79f74-64ce-4074-a1a1-d536ca5a4baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:32.794147 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa79f74-64ce-4074-a1a1-d536ca5a4baa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:32.794147 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa79f74-64ce-4074-a1a1-d536ca5a4baa admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:32.794498 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa79f74-64ce-4074-a1a1-d536ca5a4baa admin admin] PATH_INFO: `/v3/projects/9304c454ab314b68a1c0a88eee1d418c` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:32.796016 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa79f74-64ce-4074-a1a1-d536ca5a4baa admin admin] RBAC: Authorizing `identity:delete_project(project_id=9304c454ab314b68a1c0a88eee1d418c)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:32.797857 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa79f74-64ce-4074-a1a1-d536ca5a4baa admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:32.801349 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc60d4-dd44-4092-9eea-96f6241d650c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:32.806657 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bba901-7139-4541-9ea5-4e225f6ae44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:32.807439 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bba901-7139-4541-9ea5-4e225f6ae44c admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:32.807871 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bba901-7139-4541-9ea5-4e225f6ae44c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:32.808265 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bba901-7139-4541-9ea5-4e225f6ae44c admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:32.809290 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bba901-7139-4541-9ea5-4e225f6ae44c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:36:32.809693 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bba901-7139-4541-9ea5-4e225f6ae44c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:32.811930 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bba901-7139-4541-9ea5-4e225f6ae44c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:32.825929 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1276/2551] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:36:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:36:32.831422 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bba901-7139-4541-9ea5-4e225f6ae44c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:32.842141 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bba901-7139-4541-9ea5-4e225f6ae44c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:32.848684 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0fa79f74-64ce-4074-a1a1-d536ca5a4baa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9304c454ab314b68a1c0a88eee1d418c', 'request_id': 'req-0fa79f74-64ce-4074-a1a1-d536ca5a4baa'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:32.854940 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0fa79f74-64ce-4074-a1a1-d536ca5a4baa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9304c454ab314b68a1c0a88eee1d418c', 'request_id': 'req-0fa79f74-64ce-4074-a1a1-d536ca5a4baa'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:32.864208 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0fa79f74-64ce-4074-a1a1-d536ca5a4baa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9304c454ab314b68a1c0a88eee1d418c', 'request_id': 'req-0fa79f74-64ce-4074-a1a1-d536ca5a4baa'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:32.875568 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0fa79f74-64ce-4074-a1a1-d536ca5a4baa admin admin] The token cache is being invalidate because project 9304c454ab314b68a1c0a88eee1d418c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:36:32.875853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0fa79f74-64ce-4074-a1a1-d536ca5a4baa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:32.877035 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0fa79f74-64ce-4074-a1a1-d536ca5a4baa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:32.878564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1276/2552] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:36:32 2022] DELETE /identity/v3/projects/9304c454ab314b68a1c0a88eee1d418c => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:32.882481 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa79f74-64ce-4074-a1a1-d536ca5a4baa admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:32.883858 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0fa79f74-64ce-4074-a1a1-d536ca5a4baa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:32.900622 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112ecc20-908c-417d-939b-05096bfbaf4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:32.901253 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ecc20-908c-417d-939b-05096bfbaf4e admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:32.901533 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ecc20-908c-417d-939b-05096bfbaf4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:32.901833 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ecc20-908c-417d-939b-05096bfbaf4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:32.902572 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112ecc20-908c-417d-939b-05096bfbaf4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:32.904461 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112ecc20-908c-417d-939b-05096bfbaf4e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:32.939145 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1277/2553] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:36:32 2022] POST /identity/v3/projects => generated 375 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:36:32.940241 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112ecc20-908c-417d-939b-05096bfbaf4e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:32.941851 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-112ecc20-908c-417d-939b-05096bfbaf4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:32.950426 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa79f74-64ce-4074-a1a1-d536ca5a4baa admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:32.992724 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:33.008809 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b6a018-bd69-483c-94d6-c7f1ffa83c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:33.009484 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6a018-bd69-483c-94d6-c7f1ffa83c17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:33.009759 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6a018-bd69-483c-94d6-c7f1ffa83c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:33.010057 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6a018-bd69-483c-94d6-c7f1ffa83c17 admin admin] PATH_INFO: `/v3/users/9b643a52d15c4ee39ddc20f7ea8f0ba5` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:33.011377 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b6a018-bd69-483c-94d6-c7f1ffa83c17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b643a52d15c4ee39ddc20f7ea8f0ba5)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:33.011689 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112ecc20-908c-417d-939b-05096bfbaf4e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:33.013328 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b6a018-bd69-483c-94d6-c7f1ffa83c17 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:33.016207 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:33.016849 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b8f880-8f99-4c71-9b15-8351090ead51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:33.017520 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8f880-8f99-4c71-9b15-8351090ead51 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:33.017889 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8f880-8f99-4c71-9b15-8351090ead51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:33.018192 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8f880-8f99-4c71-9b15-8351090ead51 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:33.018933 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b8f880-8f99-4c71-9b15-8351090ead51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:33.021095 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b8f880-8f99-4c71-9b15-8351090ead51 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:33.044915 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1278/2554] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:36:32 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:36:33.049274 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b8f880-8f99-4c71-9b15-8351090ead51 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:33.059601 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b8f880-8f99-4c71-9b15-8351090ead51 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:33.064122 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae5b42c-cff9-40e9-9f9b-ec12904ee09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:33.064728 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5b42c-cff9-40e9-9f9b-ec12904ee09a admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:33.064853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e3b6a018-bd69-483c-94d6-c7f1ffa83c17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b643a52d15c4ee39ddc20f7ea8f0ba5'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:33.065029 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5b42c-cff9-40e9-9f9b-ec12904ee09a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:33.065270 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5b42c-cff9-40e9-9f9b-ec12904ee09a admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:33.065977 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae5b42c-cff9-40e9-9f9b-ec12904ee09a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:33.067634 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae5b42c-cff9-40e9-9f9b-ec12904ee09a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:33.073583 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e3b6a018-bd69-483c-94d6-c7f1ffa83c17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b643a52d15c4ee39ddc20f7ea8f0ba5'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:33.074270 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e3b6a018-bd69-483c-94d6-c7f1ffa83c17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b643a52d15c4ee39ddc20f7ea8f0ba5'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:33.078457 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1279/2555] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 10:36:33.082565 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae5b42c-cff9-40e9-9f9b-ec12904ee09a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:33.089785 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e3b6a018-bd69-483c-94d6-c7f1ffa83c17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b643a52d15c4ee39ddc20f7ea8f0ba5'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:33.090480 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e3b6a018-bd69-483c-94d6-c7f1ffa83c17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b643a52d15c4ee39ddc20f7ea8f0ba5'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:33.093130 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae5b42c-cff9-40e9-9f9b-ec12904ee09a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:33.097842 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94471a93-277f-4fce-a3be-92137bb37415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:33.098473 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94471a93-277f-4fce-a3be-92137bb37415 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:33.098757 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94471a93-277f-4fce-a3be-92137bb37415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:33.099060 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94471a93-277f-4fce-a3be-92137bb37415 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:33.099737 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94471a93-277f-4fce-a3be-92137bb37415 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:33.101328 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94471a93-277f-4fce-a3be-92137bb37415 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:33.103036 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1277/2556] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:36:32 2022] DELETE /identity/v3/users/9b643a52d15c4ee39ddc20f7ea8f0ba5 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:36:33.113435 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1280/2557] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 10:36:33.117739 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b6a018-bd69-483c-94d6-c7f1ffa83c17 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:33.119202 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e3b6a018-bd69-483c-94d6-c7f1ffa83c17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:33.186945 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b6a018-bd69-483c-94d6-c7f1ffa83c17 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:33.192375 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5e7cf1-8142-4375-af69-6d44867b057e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:33.193050 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e7cf1-8142-4375-af69-6d44867b057e admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:33.193326 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e7cf1-8142-4375-af69-6d44867b057e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:33.193595 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e7cf1-8142-4375-af69-6d44867b057e admin admin] PATH_INFO: `/v3/projects/47fca44a92454bb9a829a30e1afc1bd2/users/763f482ab8a34ae881d2e6101c202dac/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:33.208018 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5e7cf1-8142-4375-af69-6d44867b057e admin admin] RBAC: Authorizing `identity:create_grant(project_id=47fca44a92454bb9a829a30e1afc1bd2, user_id=763f482ab8a34ae881d2e6101c202dac, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:33.210802 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5e7cf1-8142-4375-af69-6d44867b057e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:33.224064 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1278/2558] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:36:33 2022] PUT /identity/v3/projects/47fca44a92454bb9a829a30e1afc1bd2/users/763f482ab8a34ae881d2e6101c202dac/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:36:33.234625 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90592c95-21f1-46a0-b280-2f65a08602b9 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:33.234995 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90592c95-21f1-46a0-b280-2f65a08602b9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:33.235203 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90592c95-21f1-46a0-b280-2f65a08602b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:33.271669 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-90592c95-21f1-46a0-b280-2f65a08602b9 None None] MFA Rules not processed for user `763f482ab8a34ae881d2e6101c202dac`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:33.272867 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-90592c95-21f1-46a0-b280-2f65a08602b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:33.344989 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1281/2559] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:33.356583 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5e7cf1-8142-4375-af69-6d44867b057e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:33.358332 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-dd5e7cf1-8142-4375-af69-6d44867b057e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:33.392751 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90592c95-21f1-46a0-b280-2f65a08602b9 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:33.394517 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-90592c95-21f1-46a0-b280-2f65a08602b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:33.429476 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:33.435821 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5e7cf1-8142-4375-af69-6d44867b057e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:33.442881 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2cd4e8-cb0b-486e-9ab2-a5e76abfc70b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:33.443892 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2cd4e8-cb0b-486e-9ab2-a5e76abfc70b service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:33.444377 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2cd4e8-cb0b-486e-9ab2-a5e76abfc70b service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:33.444750 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2cd4e8-cb0b-486e-9ab2-a5e76abfc70b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:33.454199 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2cd4e8-cb0b-486e-9ab2-a5e76abfc70b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:33.455914 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2cd4e8-cb0b-486e-9ab2-a5e76abfc70b service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:33.464962 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1279/2560] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:36:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:36:33.467268 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90592c95-21f1-46a0-b280-2f65a08602b9 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:33.472257 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ce16a-0953-4f72-a30a-5c610bdaf3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:33.472967 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ce16a-0953-4f72-a30a-5c610bdaf3df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:33.473311 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ce16a-0953-4f72-a30a-5c610bdaf3df admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:33.473642 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ce16a-0953-4f72-a30a-5c610bdaf3df admin admin] PATH_INFO: `/v3/projects/7699ea2d983e40e986d641915a4fee71` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:33.474885 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547ce16a-0953-4f72-a30a-5c610bdaf3df admin admin] RBAC: Authorizing `identity:delete_project(project_id=7699ea2d983e40e986d641915a4fee71)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:33.476806 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547ce16a-0953-4f72-a30a-5c610bdaf3df admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:33.528333 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:33.529833 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-547ce16a-0953-4f72-a30a-5c610bdaf3df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7699ea2d983e40e986d641915a4fee71', 'request_id': 'req-547ce16a-0953-4f72-a30a-5c610bdaf3df'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:33.542494 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-547ce16a-0953-4f72-a30a-5c610bdaf3df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7699ea2d983e40e986d641915a4fee71', 'request_id': 'req-547ce16a-0953-4f72-a30a-5c610bdaf3df'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:33.548192 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-547ce16a-0953-4f72-a30a-5c610bdaf3df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7699ea2d983e40e986d641915a4fee71', 'request_id': 'req-547ce16a-0953-4f72-a30a-5c610bdaf3df'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:33.559553 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-547ce16a-0953-4f72-a30a-5c610bdaf3df admin admin] The token cache is being invalidate because project 7699ea2d983e40e986d641915a4fee71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:36:33.559837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-547ce16a-0953-4f72-a30a-5c610bdaf3df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:33.560501 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-547ce16a-0953-4f72-a30a-5c610bdaf3df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:33.562024 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1282/2561] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:36:33 2022] DELETE /identity/v3/projects/7699ea2d983e40e986d641915a4fee71 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:36:33.566799 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002fe82-27dc-4db2-9cf2-618a513017d0 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:33.567134 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002fe82-27dc-4db2-9cf2-618a513017d0 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:33.567433 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002fe82-27dc-4db2-9cf2-618a513017d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:33.589407 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-3002fe82-27dc-4db2-9cf2-618a513017d0 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:33.589407 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3002fe82-27dc-4db2-9cf2-618a513017d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:33.679975 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1280/2562] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:33.691267 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6c275-c050-4a18-8821-37d99fb5cb8f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:33.691539 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6c275-c050-4a18-8821-37d99fb5cb8f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:33.691812 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6c275-c050-4a18-8821-37d99fb5cb8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:33.704445 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3002fe82-27dc-4db2-9cf2-618a513017d0 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:33.707684 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3002fe82-27dc-4db2-9cf2-618a513017d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:33.718891 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-fad6c275-c050-4a18-8821-37d99fb5cb8f None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:33.720123 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fad6c275-c050-4a18-8821-37d99fb5cb8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:33.720510 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3002fe82-27dc-4db2-9cf2-618a513017d0 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:33.725588 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b73392-9f1a-44d7-a672-a71a91d6e535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:33.726354 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b73392-9f1a-44d7-a672-a71a91d6e535 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:33.726699 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b73392-9f1a-44d7-a672-a71a91d6e535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:33.727140 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b73392-9f1a-44d7-a672-a71a91d6e535 admin admin] PATH_INFO: `/v3/users/763f482ab8a34ae881d2e6101c202dac` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:33.728341 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b73392-9f1a-44d7-a672-a71a91d6e535 admin admin] RBAC: Authorizing `identity:delete_user(user_id=763f482ab8a34ae881d2e6101c202dac)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:33.728974 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1283/2563] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:33.730594 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b73392-9f1a-44d7-a672-a71a91d6e535 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:33.734071 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad6c275-c050-4a18-8821-37d99fb5cb8f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:33.743245 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad6c275-c050-4a18-8821-37d99fb5cb8f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:33.748347 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ca4433-b4ea-488a-bdc3-090fec94623b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:33.748953 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ca4433-b4ea-488a-bdc3-090fec94623b admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:33.749231 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ca4433-b4ea-488a-bdc3-090fec94623b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:33.749498 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ca4433-b4ea-488a-bdc3-090fec94623b admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:33.750181 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ca4433-b4ea-488a-bdc3-090fec94623b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:36:33.750481 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ca4433-b4ea-488a-bdc3-090fec94623b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:33.752111 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ca4433-b4ea-488a-bdc3-090fec94623b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:33.765724 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1284/2564] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:36:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:36:33.770414 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ca4433-b4ea-488a-bdc3-090fec94623b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:33.780794 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ca4433-b4ea-488a-bdc3-090fec94623b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:33.785656 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac10707-fe70-43db-be17-f150ff7fbbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:33.785903 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-99b73392-9f1a-44d7-a672-a71a91d6e535 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '763f482ab8a34ae881d2e6101c202dac'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:33.786277 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac10707-fe70-43db-be17-f150ff7fbbd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:33.786554 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac10707-fe70-43db-be17-f150ff7fbbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:33.786846 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac10707-fe70-43db-be17-f150ff7fbbd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:33.787531 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac10707-fe70-43db-be17-f150ff7fbbd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:33.789447 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac10707-fe70-43db-be17-f150ff7fbbd2 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:33.795065 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-99b73392-9f1a-44d7-a672-a71a91d6e535 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '763f482ab8a34ae881d2e6101c202dac'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:33.795792 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-99b73392-9f1a-44d7-a672-a71a91d6e535 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '763f482ab8a34ae881d2e6101c202dac'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:33.807784 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-99b73392-9f1a-44d7-a672-a71a91d6e535 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '763f482ab8a34ae881d2e6101c202dac'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:33.808661 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-99b73392-9f1a-44d7-a672-a71a91d6e535 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '763f482ab8a34ae881d2e6101c202dac'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:33.820681 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1285/2565] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:36:33 2022] POST /identity/v3/projects => generated 361 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:36:33.821654 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1281/2566] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:36:33 2022] DELETE /identity/v3/users/763f482ab8a34ae881d2e6101c202dac => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:36:33.826070 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b73392-9f1a-44d7-a672-a71a91d6e535 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:33.826499 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3ba9c-be72-4122-a6cb-987666bf2a1d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:33.826783 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3ba9c-be72-4122-a6cb-987666bf2a1d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:33.827080 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3ba9c-be72-4122-a6cb-987666bf2a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:33.827659 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-99b73392-9f1a-44d7-a672-a71a91d6e535 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:33.846784 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-49f3ba9c-be72-4122-a6cb-987666bf2a1d None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:33.848300 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-49f3ba9c-be72-4122-a6cb-987666bf2a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:33.892923 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b73392-9f1a-44d7-a672-a71a91d6e535 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:33.898287 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1effc79b-7fb4-4b83-ba40-416c22dab86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:33.898991 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1effc79b-7fb4-4b83-ba40-416c22dab86b admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:33.899269 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1effc79b-7fb4-4b83-ba40-416c22dab86b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:33.899535 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1effc79b-7fb4-4b83-ba40-416c22dab86b admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:33.900191 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1effc79b-7fb4-4b83-ba40-416c22dab86b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:33.902220 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1effc79b-7fb4-4b83-ba40-416c22dab86b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:33.907148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1286/2567] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:33.911446 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:33.918626 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f3ba9c-be72-4122-a6cb-987666bf2a1d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:33.920240 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-49f3ba9c-be72-4122-a6cb-987666bf2a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:33.927687 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1282/2568] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:36:33 2022] POST /identity/v3/users => generated 281 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:36:33.932231 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1effc79b-7fb4-4b83-ba40-416c22dab86b admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:33.941686 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1effc79b-7fb4-4b83-ba40-416c22dab86b admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:33.946240 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f67e092-d85b-4777-a410-490355aa5b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:33.946822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67e092-d85b-4777-a410-490355aa5b6b admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:33.947129 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67e092-d85b-4777-a410-490355aa5b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:33.947397 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67e092-d85b-4777-a410-490355aa5b6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:33.948081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f67e092-d85b-4777-a410-490355aa5b6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:33.949705 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f67e092-d85b-4777-a410-490355aa5b6b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:33.962497 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1283/2569] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 10:36:33.963525 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f67e092-d85b-4777-a410-490355aa5b6b admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:33.973261 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f67e092-d85b-4777-a410-490355aa5b6b admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:33.977982 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a03867-e373-4341-9178-0de68ea78ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:33.978586 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a03867-e373-4341-9178-0de68ea78ed1 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:33.978759 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f3ba9c-be72-4122-a6cb-987666bf2a1d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:33.979060 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a03867-e373-4341-9178-0de68ea78ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:33.979199 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a03867-e373-4341-9178-0de68ea78ed1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:33.979811 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a03867-e373-4341-9178-0de68ea78ed1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:33.981603 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a03867-e373-4341-9178-0de68ea78ed1 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:33.983224 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9557d408-7e6a-486d-b146-eb3735df128c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:33.983853 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557d408-7e6a-486d-b146-eb3735df128c service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:33.984132 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557d408-7e6a-486d-b146-eb3735df128c service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:33.984401 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557d408-7e6a-486d-b146-eb3735df128c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:33.992641 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1284/2570] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:36:33.994959 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9557d408-7e6a-486d-b146-eb3735df128c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:33.997100 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9557d408-7e6a-486d-b146-eb3735df128c service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:34.002701 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a03867-e373-4341-9178-0de68ea78ed1 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:34.010653 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1287/2571] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:36:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:36:34.015154 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a03867-e373-4341-9178-0de68ea78ed1 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:34.019803 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e865205e-5c61-432f-8877-879e77b25035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:34.020472 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e865205e-5c61-432f-8877-879e77b25035 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:34.020748 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e865205e-5c61-432f-8877-879e77b25035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:34.021016 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e865205e-5c61-432f-8877-879e77b25035 admin admin] PATH_INFO: `/v3/projects/d69544da823d4ffba89941add977aca0/users/1e5aedba93aa4fb487a945bbf0930fc3/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:34.037167 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e865205e-5c61-432f-8877-879e77b25035 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d69544da823d4ffba89941add977aca0, user_id=1e5aedba93aa4fb487a945bbf0930fc3, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:34.039822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e865205e-5c61-432f-8877-879e77b25035 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:34.054491 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1285/2572] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:36:34 2022] PUT /identity/v3/projects/d69544da823d4ffba89941add977aca0/users/1e5aedba93aa4fb487a945bbf0930fc3/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:36:34.065348 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f97f60f-6d53-446f-94a7-ed50a5375696 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:34.065630 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f97f60f-6d53-446f-94a7-ed50a5375696 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:34.065930 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f97f60f-6d53-446f-94a7-ed50a5375696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:34.095573 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-4f97f60f-6d53-446f-94a7-ed50a5375696 None None] MFA Rules not processed for user `1e5aedba93aa4fb487a945bbf0930fc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:34.096707 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4f97f60f-6d53-446f-94a7-ed50a5375696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:34.124745 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:34.180100 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1288/2573] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:36:34.215114 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e865205e-5c61-432f-8877-879e77b25035 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:34.216877 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e865205e-5c61-432f-8877-879e77b25035 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:34.278149 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e865205e-5c61-432f-8877-879e77b25035 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:34.282284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbbd8d8-b3b2-4ab5-ae65-574b08d262ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:34.283091 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbbd8d8-b3b2-4ab5-ae65-574b08d262ea service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:34.283425 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbbd8d8-b3b2-4ab5-ae65-574b08d262ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:34.283686 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbbd8d8-b3b2-4ab5-ae65-574b08d262ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:34.294209 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f97f60f-6d53-446f-94a7-ed50a5375696 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:34.294947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbbd8d8-b3b2-4ab5-ae65-574b08d262ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:34.295960 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4f97f60f-6d53-446f-94a7-ed50a5375696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:34.296401 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbbd8d8-b3b2-4ab5-ae65-574b08d262ea service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:34.306113 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1286/2574] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:36:34 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:36:34.322201 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbbd8d8-b3b2-4ab5-ae65-574b08d262ea service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:34.366309 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f97f60f-6d53-446f-94a7-ed50a5375696 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:34.370917 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b2166-b286-4098-9a1a-cce681bf2a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:34.371574 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b2166-b286-4098-9a1a-cce681bf2a55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:34.371858 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b2166-b286-4098-9a1a-cce681bf2a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:34.372130 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b2166-b286-4098-9a1a-cce681bf2a55 admin admin] PATH_INFO: `/v3/projects/47fca44a92454bb9a829a30e1afc1bd2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:34.373299 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b2166-b286-4098-9a1a-cce681bf2a55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47fca44a92454bb9a829a30e1afc1bd2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:34.375240 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b2166-b286-4098-9a1a-cce681bf2a55 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:34.385346 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbbd8d8-b3b2-4ab5-ae65-574b08d262ea service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:34.391063 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4661ce15-d19c-467b-ac45-80bac9066aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:34.391728 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4661ce15-d19c-467b-ac45-80bac9066aec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:34.392035 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4661ce15-d19c-467b-ac45-80bac9066aec admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:34.392313 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4661ce15-d19c-467b-ac45-80bac9066aec admin admin] PATH_INFO: `/v3/users/1e5aedba93aa4fb487a945bbf0930fc3` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:34.393585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4661ce15-d19c-467b-ac45-80bac9066aec admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e5aedba93aa4fb487a945bbf0930fc3)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:34.395568 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4661ce15-d19c-467b-ac45-80bac9066aec admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:34.425605 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ac2b2166-b286-4098-9a1a-cce681bf2a55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47fca44a92454bb9a829a30e1afc1bd2', 'request_id': 'req-ac2b2166-b286-4098-9a1a-cce681bf2a55'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:34.433726 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ac2b2166-b286-4098-9a1a-cce681bf2a55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47fca44a92454bb9a829a30e1afc1bd2', 'request_id': 'req-ac2b2166-b286-4098-9a1a-cce681bf2a55'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:34.439873 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ac2b2166-b286-4098-9a1a-cce681bf2a55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47fca44a92454bb9a829a30e1afc1bd2', 'request_id': 'req-ac2b2166-b286-4098-9a1a-cce681bf2a55'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:34.443615 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4661ce15-d19c-467b-ac45-80bac9066aec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e5aedba93aa4fb487a945bbf0930fc3'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:34.446233 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:INSERT=4,keystone:SELECT=89,keystone:DELETE=7 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:34.451971 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ac2b2166-b286-4098-9a1a-cce681bf2a55 admin admin] The token cache is being invalidate because project 47fca44a92454bb9a829a30e1afc1bd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:36:34.452287 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ac2b2166-b286-4098-9a1a-cce681bf2a55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:34.453267 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ac2b2166-b286-4098-9a1a-cce681bf2a55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:34.454034 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4661ce15-d19c-467b-ac45-80bac9066aec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e5aedba93aa4fb487a945bbf0930fc3'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:34.454738 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4661ce15-d19c-467b-ac45-80bac9066aec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e5aedba93aa4fb487a945bbf0930fc3'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:34.455336 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1289/2575] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:36:34 2022] DELETE /identity/v3/projects/47fca44a92454bb9a829a30e1afc1bd2 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:34.460516 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c165a76-6ae7-41ba-b185-bd0651232227 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:34.460769 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c165a76-6ae7-41ba-b185-bd0651232227 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:34.461045 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c165a76-6ae7-41ba-b185-bd0651232227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:34.468559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4661ce15-d19c-467b-ac45-80bac9066aec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e5aedba93aa4fb487a945bbf0930fc3'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:34.469260 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4661ce15-d19c-467b-ac45-80bac9066aec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e5aedba93aa4fb487a945bbf0930fc3'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:34.481670 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1287/2576] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:36:34 2022] DELETE /identity/v3/users/1e5aedba93aa4fb487a945bbf0930fc3 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:34.482492 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-5c165a76-6ae7-41ba-b185-bd0651232227 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:34.483767 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5c165a76-6ae7-41ba-b185-bd0651232227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:34.486197 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e336b50-19ab-490e-863a-f54d3185d836 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:34.486483 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e336b50-19ab-490e-863a-f54d3185d836 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:34.486758 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e336b50-19ab-490e-863a-f54d3185d836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:34.505277 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-7e336b50-19ab-490e-863a-f54d3185d836 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:34.506469 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7e336b50-19ab-490e-863a-f54d3185d836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:34.537490 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:34.557126 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1290/2577] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:36:34.559297 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b3880-1791-4f7d-9f34-9c4fc86f9cd3 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:34.559581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b3880-1791-4f7d-9f34-9c4fc86f9cd3 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:34.559851 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b3880-1791-4f7d-9f34-9c4fc86f9cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:34.573550 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1288/2578] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:36:34.579541 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-4c0b3880-1791-4f7d-9f34-9c4fc86f9cd3 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:34.580394 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e336b50-19ab-490e-863a-f54d3185d836 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:34.580734 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4c0b3880-1791-4f7d-9f34-9c4fc86f9cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:34.581860 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7e336b50-19ab-490e-863a-f54d3185d836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:34.589052 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1291/2579] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:36:34.593336 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b3880-1791-4f7d-9f34-9c4fc86f9cd3 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:34.602229 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b3880-1791-4f7d-9f34-9c4fc86f9cd3 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:34.607107 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907eb42b-164b-4f60-84bb-a52ab356c76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:34.607704 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907eb42b-164b-4f60-84bb-a52ab356c76c admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:34.607978 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907eb42b-164b-4f60-84bb-a52ab356c76c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:34.608247 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907eb42b-164b-4f60-84bb-a52ab356c76c admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:34.608923 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907eb42b-164b-4f60-84bb-a52ab356c76c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:36:34.609222 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907eb42b-164b-4f60-84bb-a52ab356c76c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:34.610811 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907eb42b-164b-4f60-84bb-a52ab356c76c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:34.622862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1292/2580] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:36:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:36:34.627312 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907eb42b-164b-4f60-84bb-a52ab356c76c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:34.637347 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907eb42b-164b-4f60-84bb-a52ab356c76c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:34.642397 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e336b50-19ab-490e-863a-f54d3185d836 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:34.642577 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9c6946-cd49-4c3b-8715-5934f96a8ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:34.643185 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c6946-cd49-4c3b-8715-5934f96a8ead admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:34.643473 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c6946-cd49-4c3b-8715-5934f96a8ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:34.643749 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c6946-cd49-4c3b-8715-5934f96a8ead admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:34.644477 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9c6946-cd49-4c3b-8715-5934f96a8ead admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:34.646301 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9c6946-cd49-4c3b-8715-5934f96a8ead admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:34.646790 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2bb0f8-c1fe-46ae-9fe2-849834eaf99e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:34.647572 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2bb0f8-c1fe-46ae-9fe2-849834eaf99e service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:34.647845 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2bb0f8-c1fe-46ae-9fe2-849834eaf99e service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:34.648110 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2bb0f8-c1fe-46ae-9fe2-849834eaf99e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:34.658375 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2bb0f8-c1fe-46ae-9fe2-849834eaf99e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:34.659843 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2bb0f8-c1fe-46ae-9fe2-849834eaf99e service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:34.670081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1289/2581] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:36:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:36:34.676453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1293/2582] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:36:34 2022] POST /identity/v3/projects => generated 403 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:36:34.680650 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2bb0f8-c1fe-46ae-9fe2-849834eaf99e service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:34.689905 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2bb0f8-c1fe-46ae-9fe2-849834eaf99e service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:34.747877 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c0b4a4-5aa8-4669-98c1-44dd30ff2bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:34.748540 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0b4a4-5aa8-4669-98c1-44dd30ff2bc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:34.748897 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0b4a4-5aa8-4669-98c1-44dd30ff2bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:34.749146 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0b4a4-5aa8-4669-98c1-44dd30ff2bc3 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:34.749886 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c0b4a4-5aa8-4669-98c1-44dd30ff2bc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:34.751819 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c0b4a4-5aa8-4669-98c1-44dd30ff2bc3 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:34.772219 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:34.778032 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1290/2583] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:36:34 2022] POST /identity/v3/users => generated 302 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:36:34.782455 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9c6946-cd49-4c3b-8715-5934f96a8ead admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:34.793666 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9c6946-cd49-4c3b-8715-5934f96a8ead admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:34.798986 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccfb078-c3f0-49c4-bb37-01cdbc8aa8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:34.799688 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccfb078-c3f0-49c4-bb37-01cdbc8aa8d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:34.799965 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccfb078-c3f0-49c4-bb37-01cdbc8aa8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:34.800238 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccfb078-c3f0-49c4-bb37-01cdbc8aa8d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:34.800927 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccfb078-c3f0-49c4-bb37-01cdbc8aa8d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:34.802531 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccfb078-c3f0-49c4-bb37-01cdbc8aa8d8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:34.816669 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1294/2584] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:36:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:36:34.820093 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c0b4a4-5aa8-4669-98c1-44dd30ff2bc3 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:34.831885 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c0b4a4-5aa8-4669-98c1-44dd30ff2bc3 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:34.837011 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf70f2d-2779-405b-bd2a-6f1943f35e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:34.837693 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf70f2d-2779-405b-bd2a-6f1943f35e12 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:34.838032 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf70f2d-2779-405b-bd2a-6f1943f35e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:34.838368 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf70f2d-2779-405b-bd2a-6f1943f35e12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:34.839169 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf70f2d-2779-405b-bd2a-6f1943f35e12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:34.840949 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf70f2d-2779-405b-bd2a-6f1943f35e12 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:34.851652 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1291/2585] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:36:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:36:34.856150 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccfb078-c3f0-49c4-bb37-01cdbc8aa8d8 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:34.866850 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccfb078-c3f0-49c4-bb37-01cdbc8aa8d8 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:34.871578 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aac921-d13c-47dd-a695-96f23e60a9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:34.872253 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aac921-d13c-47dd-a695-96f23e60a9e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:34.872528 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aac921-d13c-47dd-a695-96f23e60a9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:34.873221 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aac921-d13c-47dd-a695-96f23e60a9e7 admin admin] PATH_INFO: `/v3/projects/dc8c2577999d43c992ec09794f81a906/users/bf4be489aee546709a4ebc7e4ac26c3d/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:34.888657 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aac921-d13c-47dd-a695-96f23e60a9e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc8c2577999d43c992ec09794f81a906, user_id=bf4be489aee546709a4ebc7e4ac26c3d, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:34.891693 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aac921-d13c-47dd-a695-96f23e60a9e7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:34.917422 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1295/2586] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:36:34 2022] PUT /identity/v3/projects/dc8c2577999d43c992ec09794f81a906/users/bf4be489aee546709a4ebc7e4ac26c3d/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:36:34.919480 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d389f20-36f8-4d02-b790-b5b9517ad77e None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:34.919814 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d389f20-36f8-4d02-b790-b5b9517ad77e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:34.920080 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d389f20-36f8-4d02-b790-b5b9517ad77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:34.948581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aac921-d13c-47dd-a695-96f23e60a9e7 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:34.950728 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-71aac921-d13c-47dd-a695-96f23e60a9e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:34.952727 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-9d389f20-36f8-4d02-b790-b5b9517ad77e None None] MFA Rules not processed for user `bf4be489aee546709a4ebc7e4ac26c3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:34.954334 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9d389f20-36f8-4d02-b790-b5b9517ad77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:35.024230 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1292/2587] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:34 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:35.027661 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aac921-d13c-47dd-a695-96f23e60a9e7 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:35.029216 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d389f20-36f8-4d02-b790-b5b9517ad77e None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:35.033455 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdf1ee3-dad7-4a65-bcfd-914f548db5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:35.034135 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf1ee3-dad7-4a65-bcfd-914f548db5fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:35.034621 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf1ee3-dad7-4a65-bcfd-914f548db5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:35.034913 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf1ee3-dad7-4a65-bcfd-914f548db5fa admin admin] PATH_INFO: `/v3/projects/d69544da823d4ffba89941add977aca0` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:35.036099 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdf1ee3-dad7-4a65-bcfd-914f548db5fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=d69544da823d4ffba89941add977aca0)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:35.037987 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdf1ee3-dad7-4a65-bcfd-914f548db5fa admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:35.039616 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d389f20-36f8-4d02-b790-b5b9517ad77e None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:35.044264 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f44058-f965-4ca7-924c-512d8e67c153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:35.044988 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f44058-f965-4ca7-924c-512d8e67c153 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:35.045365 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f44058-f965-4ca7-924c-512d8e67c153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:35.045642 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f44058-f965-4ca7-924c-512d8e67c153 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:35.046396 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f44058-f965-4ca7-924c-512d8e67c153 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:35.048415 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f44058-f965-4ca7-924c-512d8e67c153 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:35.078225 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1293/2588] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:36:35 2022] POST /identity/v3/projects => generated 403 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:36:35.082662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f44058-f965-4ca7-924c-512d8e67c153 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:35.088027 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1bdf1ee3-dad7-4a65-bcfd-914f548db5fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd69544da823d4ffba89941add977aca0', 'request_id': 'req-1bdf1ee3-dad7-4a65-bcfd-914f548db5fa'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:35.097834 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1bdf1ee3-dad7-4a65-bcfd-914f548db5fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd69544da823d4ffba89941add977aca0', 'request_id': 'req-1bdf1ee3-dad7-4a65-bcfd-914f548db5fa'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:35.104353 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1bdf1ee3-dad7-4a65-bcfd-914f548db5fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd69544da823d4ffba89941add977aca0', 'request_id': 'req-1bdf1ee3-dad7-4a65-bcfd-914f548db5fa'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:35.116308 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1bdf1ee3-dad7-4a65-bcfd-914f548db5fa admin admin] The token cache is being invalidate because project d69544da823d4ffba89941add977aca0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:36:35.116598 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1bdf1ee3-dad7-4a65-bcfd-914f548db5fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:35.117255 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1bdf1ee3-dad7-4a65-bcfd-914f548db5fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:35.118791 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1296/2589] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:36:34 2022] DELETE /identity/v3/projects/d69544da823d4ffba89941add977aca0 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:35.124113 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfef2d9-11e8-459a-a415-c42f20a45b8a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:35.124388 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfef2d9-11e8-459a-a415-c42f20a45b8a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:35.124740 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfef2d9-11e8-459a-a415-c42f20a45b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:35.144493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ecfef2d9-11e8-459a-a415-c42f20a45b8a None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:35.145711 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ecfef2d9-11e8-459a-a415-c42f20a45b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:35.146991 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f44058-f965-4ca7-924c-512d8e67c153 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:35.155272 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1297/2590] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:36:35.159457 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf61b25-dad8-4f5b-8cd6-c74e79076b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:35.160141 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf61b25-dad8-4f5b-8cd6-c74e79076b92 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:35.160500 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf61b25-dad8-4f5b-8cd6-c74e79076b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:35.160833 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf61b25-dad8-4f5b-8cd6-c74e79076b92 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:35.161609 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf61b25-dad8-4f5b-8cd6-c74e79076b92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:35.163353 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc07b9-5315-4a69-9ac7-f2918649418e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:35.163799 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf61b25-dad8-4f5b-8cd6-c74e79076b92 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:35.163945 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc07b9-5315-4a69-9ac7-f2918649418e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:35.164173 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc07b9-5315-4a69-9ac7-f2918649418e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:35.173018 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:35.186360 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-14cc07b9-5315-4a69-9ac7-f2918649418e None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:35.187825 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1294/2591] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:36:35 2022] POST /identity/v3/users => generated 308 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:36:35.188213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-14cc07b9-5315-4a69-9ac7-f2918649418e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:35.191700 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf61b25-dad8-4f5b-8cd6-c74e79076b92 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:35.193466 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9cf61b25-dad8-4f5b-8cd6-c74e79076b92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:35.196509 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1298/2592] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:36:35.200608 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cc07b9-5315-4a69-9ac7-f2918649418e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:35.206255 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf61b25-dad8-4f5b-8cd6-c74e79076b92 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:35.209356 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cc07b9-5315-4a69-9ac7-f2918649418e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:35.211211 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8556fd0-da04-435a-a70f-da3373f74f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:35.211825 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8556fd0-da04-435a-a70f-da3373f74f7d admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:35.212112 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8556fd0-da04-435a-a70f-da3373f74f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:35.212368 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8556fd0-da04-435a-a70f-da3373f74f7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:35.213095 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8556fd0-da04-435a-a70f-da3373f74f7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:35.214649 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9b4961-4920-4902-8b3b-6aec1c656fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:35.214806 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8556fd0-da04-435a-a70f-da3373f74f7d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:35.215275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b4961-4920-4902-8b3b-6aec1c656fb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:35.215570 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b4961-4920-4902-8b3b-6aec1c656fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:35.215837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b4961-4920-4902-8b3b-6aec1c656fb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:35.216601 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9b4961-4920-4902-8b3b-6aec1c656fb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:36:35.216964 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9b4961-4920-4902-8b3b-6aec1c656fb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:35.218638 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9b4961-4920-4902-8b3b-6aec1c656fb8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:35.225207 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1295/2593] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:36:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:36:35.229550 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8556fd0-da04-435a-a70f-da3373f74f7d admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:35.231848 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1299/2594] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:36:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:36:35.236091 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9b4961-4920-4902-8b3b-6aec1c656fb8 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:35.239606 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8556fd0-da04-435a-a70f-da3373f74f7d admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:35.244249 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0ed5c6-de7e-4a0b-a34c-6980d5443ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:35.244907 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ed5c6-de7e-4a0b-a34c-6980d5443ead admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:35.245210 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ed5c6-de7e-4a0b-a34c-6980d5443ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:35.245524 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ed5c6-de7e-4a0b-a34c-6980d5443ead admin admin] PATH_INFO: `/v3/projects/c731a5f4c3ae4680a504bbc7d31d16cc/users/096561b0200349eba48e7b9e28984222/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:35.246058 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9b4961-4920-4902-8b3b-6aec1c656fb8 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:35.250718 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f658679-4540-4a3e-a0e1-0e5e87c232e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:35.251343 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f658679-4540-4a3e-a0e1-0e5e87c232e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:35.251640 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f658679-4540-4a3e-a0e1-0e5e87c232e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:35.251914 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f658679-4540-4a3e-a0e1-0e5e87c232e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:35.252569 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f658679-4540-4a3e-a0e1-0e5e87c232e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:35.254467 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f658679-4540-4a3e-a0e1-0e5e87c232e0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:35.260510 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0ed5c6-de7e-4a0b-a34c-6980d5443ead admin admin] RBAC: Authorizing `identity:create_grant(project_id=c731a5f4c3ae4680a504bbc7d31d16cc, user_id=096561b0200349eba48e7b9e28984222, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:35.263381 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0ed5c6-de7e-4a0b-a34c-6980d5443ead admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:35.278537 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1296/2595] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:36:35 2022] PUT /identity/v3/projects/c731a5f4c3ae4680a504bbc7d31d16cc/users/096561b0200349eba48e7b9e28984222/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:36:35.282518 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0ed5c6-de7e-4a0b-a34c-6980d5443ead admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:35.286054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1300/2596] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:36:35 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:36:35.287121 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f658679-4540-4a3e-a0e1-0e5e87c232e0 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:35.294644 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:35.345681 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0ed5c6-de7e-4a0b-a34c-6980d5443ead admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:35.350056 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f658679-4540-4a3e-a0e1-0e5e87c232e0 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:35.351304 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228402df-2ab2-408b-9e89-540adb0667ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:35.351894 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228402df-2ab2-408b-9e89-540adb0667ea admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:35.352173 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228402df-2ab2-408b-9e89-540adb0667ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:35.352440 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228402df-2ab2-408b-9e89-540adb0667ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:35.353161 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228402df-2ab2-408b-9e89-540adb0667ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:35.354980 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228402df-2ab2-408b-9e89-540adb0667ea admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:35.355291 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371501b3-5176-4f57-a185-6791891c1e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:35.355914 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371501b3-5176-4f57-a185-6791891c1e25 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:35.356198 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371501b3-5176-4f57-a185-6791891c1e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:35.356472 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371501b3-5176-4f57-a185-6791891c1e25 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:35.357196 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371501b3-5176-4f57-a185-6791891c1e25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:35.359196 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371501b3-5176-4f57-a185-6791891c1e25 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:35.365033 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1297/2597] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:36:35 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 10:36:35.369307 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228402df-2ab2-408b-9e89-540adb0667ea admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:35.379149 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228402df-2ab2-408b-9e89-540adb0667ea admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:35.384495 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1301/2598] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:36:35 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:36:35.384640 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6459ed-cc4d-4c0d-b796-745c513c78ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:35.385051 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6459ed-cc4d-4c0d-b796-745c513c78ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:35.385336 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6459ed-cc4d-4c0d-b796-745c513c78ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:35.385463 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371501b3-5176-4f57-a185-6791891c1e25 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:35.385629 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6459ed-cc4d-4c0d-b796-745c513c78ec admin admin] PATH_INFO: `/v3/projects/c731a5f4c3ae4680a504bbc7d31d16cc/users/096561b0200349eba48e7b9e28984222/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:35.387676 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6459ed-cc4d-4c0d-b796-745c513c78ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=c731a5f4c3ae4680a504bbc7d31d16cc, user_id=096561b0200349eba48e7b9e28984222, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:35.390638 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6459ed-cc4d-4c0d-b796-745c513c78ec admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:35.396088 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371501b3-5176-4f57-a185-6791891c1e25 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:35.401642 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1298/2599] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:36:35 2022] PUT /identity/v3/projects/c731a5f4c3ae4680a504bbc7d31d16cc/users/096561b0200349eba48e7b9e28984222/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:36:35.412058 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730431c-88e2-42ed-b7e0-d06c38c89430 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:35.412344 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730431c-88e2-42ed-b7e0-d06c38c89430 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:35.412645 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730431c-88e2-42ed-b7e0-d06c38c89430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:35.440745 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-9730431c-88e2-42ed-b7e0-d06c38c89430 None None] MFA Rules not processed for user `096561b0200349eba48e7b9e28984222`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:35.441967 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9730431c-88e2-42ed-b7e0-d06c38c89430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:35.454866 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dade03b-b6e8-4f0e-a105-bcaba6c8b384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:35.455522 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dade03b-b6e8-4f0e-a105-bcaba6c8b384 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:35.455805 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dade03b-b6e8-4f0e-a105-bcaba6c8b384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:35.456081 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dade03b-b6e8-4f0e-a105-bcaba6c8b384 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:35.456775 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dade03b-b6e8-4f0e-a105-bcaba6c8b384 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:35.458385 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dade03b-b6e8-4f0e-a105-bcaba6c8b384 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:35.468556 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1302/2600] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:36:35 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 10:36:35.472685 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dade03b-b6e8-4f0e-a105-bcaba6c8b384 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:35.482764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dade03b-b6e8-4f0e-a105-bcaba6c8b384 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:35.488198 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17191a96-c078-496f-982b-a69b0f94ed9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:35.488862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17191a96-c078-496f-982b-a69b0f94ed9a admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:35.489152 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17191a96-c078-496f-982b-a69b0f94ed9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:35.489428 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17191a96-c078-496f-982b-a69b0f94ed9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:35.490130 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17191a96-c078-496f-982b-a69b0f94ed9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:35.491755 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17191a96-c078-496f-982b-a69b0f94ed9a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:35.503575 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1303/2601] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:36:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 10:36:35.507628 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17191a96-c078-496f-982b-a69b0f94ed9a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:35.511893 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1299/2602] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:35 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:36:35.518134 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17191a96-c078-496f-982b-a69b0f94ed9a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:35.522634 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9730431c-88e2-42ed-b7e0-d06c38c89430 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:35.522891 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5918ca-3782-418a-8264-f6d3130ed212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:35.523580 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5918ca-3782-418a-8264-f6d3130ed212 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:35.523862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5918ca-3782-418a-8264-f6d3130ed212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:35.523971 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9730431c-88e2-42ed-b7e0-d06c38c89430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:35.524139 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5918ca-3782-418a-8264-f6d3130ed212 admin admin] PATH_INFO: `/v3/projects/f5c7be2e565d44cebf1aba36f7b3720c/users/afafb85b5e17441c845aaacecaca3ca8/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:35.538868 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5918ca-3782-418a-8264-f6d3130ed212 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5c7be2e565d44cebf1aba36f7b3720c, user_id=afafb85b5e17441c845aaacecaca3ca8, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:35.541538 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5918ca-3782-418a-8264-f6d3130ed212 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:35.556241 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1304/2603] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:36:35 2022] PUT /identity/v3/projects/f5c7be2e565d44cebf1aba36f7b3720c/users/afafb85b5e17441c845aaacecaca3ca8/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:36:35.585669 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9730431c-88e2-42ed-b7e0-d06c38c89430 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:35.622629 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:35.626064 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5e4d6-c912-457c-877e-c30cb2e53866 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:35.626425 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5e4d6-c912-457c-877e-c30cb2e53866 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:35.626651 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5e4d6-c912-457c-877e-c30cb2e53866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:35.643012 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33817d23-8af0-4035-8a88-08c5aa9a5aaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:35.643557 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33817d23-8af0-4035-8a88-08c5aa9a5aaf service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:35.643849 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33817d23-8af0-4035-8a88-08c5aa9a5aaf service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:35.644103 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33817d23-8af0-4035-8a88-08c5aa9a5aaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:35.652691 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:35.656125 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-58c5e4d6-c912-457c-877e-c30cb2e53866 None None] MFA Rules not processed for user `afafb85b5e17441c845aaacecaca3ca8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:35.658049 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-58c5e4d6-c912-457c-877e-c30cb2e53866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:35.703990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33817d23-8af0-4035-8a88-08c5aa9a5aaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:35.705445 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33817d23-8af0-4035-8a88-08c5aa9a5aaf service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:35.715494 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1300/2604] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:36:35 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 10:36:35.722411 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1305/2605] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:35.736008 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33817d23-8af0-4035-8a88-08c5aa9a5aaf service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:35.737776 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-33817d23-8af0-4035-8a88-08c5aa9a5aaf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:35.750203 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33817d23-8af0-4035-8a88-08c5aa9a5aaf service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:35.754243 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f81b5-6ee1-4bac-bf0c-536f8d8e6358 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:35.755022 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f81b5-6ee1-4bac-bf0c-536f8d8e6358 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:35.755213 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f81b5-6ee1-4bac-bf0c-536f8d8e6358 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:35.755480 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f81b5-6ee1-4bac-bf0c-536f8d8e6358 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:35.764906 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0f81b5-6ee1-4bac-bf0c-536f8d8e6358 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:35.766330 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0f81b5-6ee1-4bac-bf0c-536f8d8e6358 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:35.776192 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1301/2606] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:36:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:36:35.871472 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c5e4d6-c912-457c-877e-c30cb2e53866 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:35.940884 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c5e4d6-c912-457c-877e-c30cb2e53866 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:35.945419 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92acfb6-91da-4702-909e-cd8e8911adc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:35.946063 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92acfb6-91da-4702-909e-cd8e8911adc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:35.946333 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92acfb6-91da-4702-909e-cd8e8911adc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:35.946605 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92acfb6-91da-4702-909e-cd8e8911adc3 admin admin] PATH_INFO: `/v3/users/bf4be489aee546709a4ebc7e4ac26c3d` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:35.947935 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92acfb6-91da-4702-909e-cd8e8911adc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf4be489aee546709a4ebc7e4ac26c3d)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:35.950024 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92acfb6-91da-4702-909e-cd8e8911adc3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:36.000142 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b92acfb6-91da-4702-909e-cd8e8911adc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf4be489aee546709a4ebc7e4ac26c3d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:36.000800 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b92acfb6-91da-4702-909e-cd8e8911adc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf4be489aee546709a4ebc7e4ac26c3d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:36.001412 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b92acfb6-91da-4702-909e-cd8e8911adc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf4be489aee546709a4ebc7e4ac26c3d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:36.010819 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b92acfb6-91da-4702-909e-cd8e8911adc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf4be489aee546709a4ebc7e4ac26c3d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:36.022633 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b92acfb6-91da-4702-909e-cd8e8911adc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf4be489aee546709a4ebc7e4ac26c3d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:36.051947 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1306/2607] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:36:35 2022] DELETE /identity/v3/users/bf4be489aee546709a4ebc7e4ac26c3d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:36.053708 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3398a1e5-f043-4709-82db-610e47fca34c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:36.053995 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3398a1e5-f043-4709-82db-610e47fca34c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:36.054278 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3398a1e5-f043-4709-82db-610e47fca34c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:36.075361 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-3398a1e5-f043-4709-82db-610e47fca34c None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:36.076924 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3398a1e5-f043-4709-82db-610e47fca34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:36.137458 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1302/2608] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:36.147685 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92acfb6-91da-4702-909e-cd8e8911adc3 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:36.149193 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b92acfb6-91da-4702-909e-cd8e8911adc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:36.176200 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:36.210453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92acfb6-91da-4702-909e-cd8e8911adc3 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:36.214507 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee4f7f-4208-431a-a16c-199b76c7c265 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:36.215251 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee4f7f-4208-431a-a16c-199b76c7c265 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:36.215525 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee4f7f-4208-431a-a16c-199b76c7c265 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:36.215795 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee4f7f-4208-431a-a16c-199b76c7c265 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:36.225281 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ee4f7f-4208-431a-a16c-199b76c7c265 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:36.226759 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ee4f7f-4208-431a-a16c-199b76c7c265 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:36.236697 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1307/2609] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:36:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:36:36.506635 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3398a1e5-f043-4709-82db-610e47fca34c None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:36.508379 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3398a1e5-f043-4709-82db-610e47fca34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:36.526480 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3398a1e5-f043-4709-82db-610e47fca34c None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:36.532355 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb0155-aa23-4798-8335-14876acb00b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:36.533161 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb0155-aa23-4798-8335-14876acb00b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:36.533344 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb0155-aa23-4798-8335-14876acb00b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:36.533684 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb0155-aa23-4798-8335-14876acb00b4 admin admin] PATH_INFO: `/v3/projects/dc8c2577999d43c992ec09794f81a906` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:36.536902 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbb0155-aa23-4798-8335-14876acb00b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc8c2577999d43c992ec09794f81a906)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:36.538918 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbb0155-aa23-4798-8335-14876acb00b4 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:36.588791 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3dbb0155-aa23-4798-8335-14876acb00b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc8c2577999d43c992ec09794f81a906', 'request_id': 'req-3dbb0155-aa23-4798-8335-14876acb00b4'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:36.594544 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3dbb0155-aa23-4798-8335-14876acb00b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc8c2577999d43c992ec09794f81a906', 'request_id': 'req-3dbb0155-aa23-4798-8335-14876acb00b4'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:36.604411 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3dbb0155-aa23-4798-8335-14876acb00b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc8c2577999d43c992ec09794f81a906', 'request_id': 'req-3dbb0155-aa23-4798-8335-14876acb00b4'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:36.606895 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:36.617364 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3dbb0155-aa23-4798-8335-14876acb00b4 admin admin] The token cache is being invalidate because project dc8c2577999d43c992ec09794f81a906 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:36:36.617651 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3dbb0155-aa23-4798-8335-14876acb00b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:36.618331 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3dbb0155-aa23-4798-8335-14876acb00b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:36.619948 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1303/2610] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:36:36 2022] DELETE /identity/v3/projects/dc8c2577999d43c992ec09794f81a906 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:36.624362 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee4f7f-4208-431a-a16c-199b76c7c265 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:36.634545 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee4f7f-4208-431a-a16c-199b76c7c265 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:36.700308 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc94989-cb3e-4ec8-8f34-ea65b7596707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:36.700945 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc94989-cb3e-4ec8-8f34-ea65b7596707 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:36.701228 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc94989-cb3e-4ec8-8f34-ea65b7596707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:36.701496 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc94989-cb3e-4ec8-8f34-ea65b7596707 admin admin] PATH_INFO: `/v3/users/096561b0200349eba48e7b9e28984222` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:36.702736 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc94989-cb3e-4ec8-8f34-ea65b7596707 admin admin] RBAC: Authorizing `identity:delete_user(user_id=096561b0200349eba48e7b9e28984222)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:36.704695 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc94989-cb3e-4ec8-8f34-ea65b7596707 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:36.758125 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1cc94989-cb3e-4ec8-8f34-ea65b7596707 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '096561b0200349eba48e7b9e28984222'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:36.758749 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1cc94989-cb3e-4ec8-8f34-ea65b7596707 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '096561b0200349eba48e7b9e28984222'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:36.759409 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1cc94989-cb3e-4ec8-8f34-ea65b7596707 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '096561b0200349eba48e7b9e28984222'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:36.769283 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1cc94989-cb3e-4ec8-8f34-ea65b7596707 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '096561b0200349eba48e7b9e28984222'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:36.776379 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1cc94989-cb3e-4ec8-8f34-ea65b7596707 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '096561b0200349eba48e7b9e28984222'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:36.796061 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1308/2611] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:36:36 2022] DELETE /identity/v3/users/096561b0200349eba48e7b9e28984222 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:37.074026 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb0155-aa23-4798-8335-14876acb00b4 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:37.075640 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3dbb0155-aa23-4798-8335-14876acb00b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:37.092179 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb0155-aa23-4798-8335-14876acb00b4 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:37.166225 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41daaccf-7e2d-43e0-88d4-d23228d1eed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:37.166995 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41daaccf-7e2d-43e0-88d4-d23228d1eed2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:37.167418 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41daaccf-7e2d-43e0-88d4-d23228d1eed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:37.167753 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41daaccf-7e2d-43e0-88d4-d23228d1eed2 admin admin] PATH_INFO: `/v3/projects/c731a5f4c3ae4680a504bbc7d31d16cc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:37.169109 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41daaccf-7e2d-43e0-88d4-d23228d1eed2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c731a5f4c3ae4680a504bbc7d31d16cc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:37.171259 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41daaccf-7e2d-43e0-88d4-d23228d1eed2 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:37.225861 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-41daaccf-7e2d-43e0-88d4-d23228d1eed2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c731a5f4c3ae4680a504bbc7d31d16cc', 'request_id': 'req-41daaccf-7e2d-43e0-88d4-d23228d1eed2'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:37.232058 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-41daaccf-7e2d-43e0-88d4-d23228d1eed2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c731a5f4c3ae4680a504bbc7d31d16cc', 'request_id': 'req-41daaccf-7e2d-43e0-88d4-d23228d1eed2'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:37.240835 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-41daaccf-7e2d-43e0-88d4-d23228d1eed2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c731a5f4c3ae4680a504bbc7d31d16cc', 'request_id': 'req-41daaccf-7e2d-43e0-88d4-d23228d1eed2'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:37.252916 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-41daaccf-7e2d-43e0-88d4-d23228d1eed2 admin admin] The token cache is being invalidate because project c731a5f4c3ae4680a504bbc7d31d16cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:36:37.253209 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-41daaccf-7e2d-43e0-88d4-d23228d1eed2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:37.253878 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-41daaccf-7e2d-43e0-88d4-d23228d1eed2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:37.255470 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1304/2612] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:36:37 2022] DELETE /identity/v3/projects/c731a5f4c3ae4680a504bbc7d31d16cc => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:37.260508 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b5117f-5d7b-44d3-ba7f-acc984c808da None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:37.261100 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b5117f-5d7b-44d3-ba7f-acc984c808da None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:37.261197 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b5117f-5d7b-44d3-ba7f-acc984c808da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:37.282184 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-72b5117f-5d7b-44d3-ba7f-acc984c808da None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:37.283524 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-72b5117f-5d7b-44d3-ba7f-acc984c808da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:37.324999 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:37.345510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1309/2613] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:37.354889 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb04dde-14f6-441e-b808-56a949cb6cb1 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:37.355317 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb04dde-14f6-441e-b808-56a949cb6cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:37.355593 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb04dde-14f6-441e-b808-56a949cb6cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:37.376123 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-edb04dde-14f6-441e-b808-56a949cb6cb1 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:37.377323 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-edb04dde-14f6-441e-b808-56a949cb6cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:37.391410 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b5117f-5d7b-44d3-ba7f-acc984c808da None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:37.394973 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1305/2614] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:37.401240 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b5117f-5d7b-44d3-ba7f-acc984c808da None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:37.406198 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e82115-f9a1-45c8-a22a-27f6802da358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:37.406815 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e82115-f9a1-45c8-a22a-27f6802da358 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:37.407106 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e82115-f9a1-45c8-a22a-27f6802da358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:37.407384 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e82115-f9a1-45c8-a22a-27f6802da358 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:37.408076 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e82115-f9a1-45c8-a22a-27f6802da358 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:36:37.408546 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e82115-f9a1-45c8-a22a-27f6802da358 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:37.409953 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e82115-f9a1-45c8-a22a-27f6802da358 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:37.423071 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1310/2615] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:36:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:36:37.426717 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb04dde-14f6-441e-b808-56a949cb6cb1 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:37.437161 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb04dde-14f6-441e-b808-56a949cb6cb1 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:37.441955 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb890284-1889-4091-b556-20ab0448f979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:37.442647 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb890284-1889-4091-b556-20ab0448f979 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:37.443066 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb890284-1889-4091-b556-20ab0448f979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:37.443402 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb890284-1889-4091-b556-20ab0448f979 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:37.444168 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb890284-1889-4091-b556-20ab0448f979 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:37.446426 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb890284-1889-4091-b556-20ab0448f979 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:37.477920 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1306/2616] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:36:37 2022] POST /identity/v3/projects => generated 363 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:36:37.482012 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e82115-f9a1-45c8-a22a-27f6802da358 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:37.492544 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e82115-f9a1-45c8-a22a-27f6802da358 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:37.552439 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e7653-7004-448d-9948-fe1c675d9bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:37.553147 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e7653-7004-448d-9948-fe1c675d9bd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:37.553538 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e7653-7004-448d-9948-fe1c675d9bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:37.553719 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e7653-7004-448d-9948-fe1c675d9bd5 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:37.554354 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107e7653-7004-448d-9948-fe1c675d9bd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:37.556316 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107e7653-7004-448d-9948-fe1c675d9bd5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:37.582769 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1311/2617] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:36:37 2022] POST /identity/v3/users => generated 282 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:36:37.586891 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb890284-1889-4091-b556-20ab0448f979 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:37.597492 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb890284-1889-4091-b556-20ab0448f979 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:37.602126 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c2f50-215d-41e9-9650-81cdae526d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:37.602744 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c2f50-215d-41e9-9650-81cdae526d44 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:37.603057 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c2f50-215d-41e9-9650-81cdae526d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:37.603334 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c2f50-215d-41e9-9650-81cdae526d44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:37.604037 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92c2f50-215d-41e9-9650-81cdae526d44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:37.605829 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92c2f50-215d-41e9-9650-81cdae526d44 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:37.617029 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1307/2618] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:36:37.621168 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e7653-7004-448d-9948-fe1c675d9bd5 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:37.631483 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e7653-7004-448d-9948-fe1c675d9bd5 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:37.636102 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3952e8-149a-49fa-9000-3a54639c1199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:37.636711 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3952e8-149a-49fa-9000-3a54639c1199 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:37.636984 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3952e8-149a-49fa-9000-3a54639c1199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:37.637251 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3952e8-149a-49fa-9000-3a54639c1199 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:37.637931 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3952e8-149a-49fa-9000-3a54639c1199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:37.639896 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3952e8-149a-49fa-9000-3a54639c1199 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:37.654973 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1312/2619] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:36:37.658481 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c2f50-215d-41e9-9650-81cdae526d44 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:37.670284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c2f50-215d-41e9-9650-81cdae526d44 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:37.675060 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d22aab-ea02-4109-a8e2-25654f19dd9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:37.675734 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d22aab-ea02-4109-a8e2-25654f19dd9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:37.676016 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d22aab-ea02-4109-a8e2-25654f19dd9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:37.676333 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d22aab-ea02-4109-a8e2-25654f19dd9a admin admin] PATH_INFO: `/v3/projects/f001bcce76084f89a4efe99550e5bc80/users/5e67ca02d15e4a20b972630b7fd02335/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:37.691164 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d22aab-ea02-4109-a8e2-25654f19dd9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f001bcce76084f89a4efe99550e5bc80, user_id=5e67ca02d15e4a20b972630b7fd02335, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:37.694066 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d22aab-ea02-4109-a8e2-25654f19dd9a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:37.714455 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1308/2620] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:36:37 2022] PUT /identity/v3/projects/f001bcce76084f89a4efe99550e5bc80/users/5e67ca02d15e4a20b972630b7fd02335/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:37.725568 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8df9b-b8e5-4f6d-8df6-25c4c093d895 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:37.725568 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8df9b-b8e5-4f6d-8df6-25c4c093d895 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:37.725839 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8df9b-b8e5-4f6d-8df6-25c4c093d895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:37.755530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-cfc8df9b-b8e5-4f6d-8df6-25c4c093d895 None None] MFA Rules not processed for user `5e67ca02d15e4a20b972630b7fd02335`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:37.756848 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cfc8df9b-b8e5-4f6d-8df6-25c4c093d895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:37.828047 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1313/2621] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:37.832617 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d22aab-ea02-4109-a8e2-25654f19dd9a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:37.858927 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:37.898212 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d22aab-ea02-4109-a8e2-25654f19dd9a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:37.904086 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253e32b0-9aee-40f5-b803-c37d997cce94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:37.904703 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e32b0-9aee-40f5-b803-c37d997cce94 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:37.904977 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e32b0-9aee-40f5-b803-c37d997cce94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:37.905248 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e32b0-9aee-40f5-b803-c37d997cce94 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:37.905941 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253e32b0-9aee-40f5-b803-c37d997cce94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:37.907872 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253e32b0-9aee-40f5-b803-c37d997cce94 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:37.942297 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1309/2622] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:36:37 2022] POST /identity/v3/projects => generated 363 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:36:37.948039 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc8df9b-b8e5-4f6d-8df6-25c4c093d895 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:37.959597 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc8df9b-b8e5-4f6d-8df6-25c4c093d895 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:38.004579 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:38.027811 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a77d368-cd9c-4077-a4de-6d53e1d774e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:38.028364 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77d368-cd9c-4077-a4de-6d53e1d774e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:38.028816 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77d368-cd9c-4077-a4de-6d53e1d774e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:38.029022 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77d368-cd9c-4077-a4de-6d53e1d774e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:38.029782 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a77d368-cd9c-4077-a4de-6d53e1d774e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:38.031858 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a77d368-cd9c-4077-a4de-6d53e1d774e9 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:38.055212 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1314/2623] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:36:37 2022] POST /identity/v3/users => generated 288 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:36:38.059473 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253e32b0-9aee-40f5-b803-c37d997cce94 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:38.069947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253e32b0-9aee-40f5-b803-c37d997cce94 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:38.074927 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0862e958-f374-4fdf-a2f4-aacaddb0c42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:38.075564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0862e958-f374-4fdf-a2f4-aacaddb0c42c admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:38.075819 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0862e958-f374-4fdf-a2f4-aacaddb0c42c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:38.076121 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0862e958-f374-4fdf-a2f4-aacaddb0c42c admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:38.076817 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0862e958-f374-4fdf-a2f4-aacaddb0c42c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:38.078426 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0862e958-f374-4fdf-a2f4-aacaddb0c42c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:38.089867 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1310/2624] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:36:38.094237 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a77d368-cd9c-4077-a4de-6d53e1d774e9 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:38.104883 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a77d368-cd9c-4077-a4de-6d53e1d774e9 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:38.109775 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bba65f-8757-4e90-b2bc-c2aa62b51697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:38.110452 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bba65f-8757-4e90-b2bc-c2aa62b51697 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:38.110726 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bba65f-8757-4e90-b2bc-c2aa62b51697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:38.111031 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bba65f-8757-4e90-b2bc-c2aa62b51697 admin admin] PATH_INFO: `/v3/projects/b81d1f86192240f493fdeb9b4badac8d/users/d464a830754344a5ba9f47380dc72801/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:38.127908 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bba65f-8757-4e90-b2bc-c2aa62b51697 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b81d1f86192240f493fdeb9b4badac8d, user_id=d464a830754344a5ba9f47380dc72801, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:38.130565 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bba65f-8757-4e90-b2bc-c2aa62b51697 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:38.145225 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1315/2625] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:36:38 2022] PUT /identity/v3/projects/b81d1f86192240f493fdeb9b4badac8d/users/d464a830754344a5ba9f47380dc72801/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:38.149066 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0862e958-f374-4fdf-a2f4-aacaddb0c42c admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:38.159690 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0862e958-f374-4fdf-a2f4-aacaddb0c42c admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:38.218672 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6c015-d05f-46fd-a577-cf76d6b1a9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:38.219769 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6c015-d05f-46fd-a577-cf76d6b1a9b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:38.220229 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6c015-d05f-46fd-a577-cf76d6b1a9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:38.220666 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6c015-d05f-46fd-a577-cf76d6b1a9b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:38.221748 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be6c015-d05f-46fd-a577-cf76d6b1a9b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:38.224097 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be6c015-d05f-46fd-a577-cf76d6b1a9b2 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:38.238795 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1311/2626] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:36:38.245615 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bba65f-8757-4e90-b2bc-c2aa62b51697 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:38.257065 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bba65f-8757-4e90-b2bc-c2aa62b51697 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:38.262628 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd68aa5c-0652-4c19-93cf-bb6f0d7455d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:38.263386 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd68aa5c-0652-4c19-93cf-bb6f0d7455d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:38.263672 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd68aa5c-0652-4c19-93cf-bb6f0d7455d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:38.263942 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd68aa5c-0652-4c19-93cf-bb6f0d7455d0 admin admin] PATH_INFO: `/v3/projects/b81d1f86192240f493fdeb9b4badac8d/users/d464a830754344a5ba9f47380dc72801/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:38.266382 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd68aa5c-0652-4c19-93cf-bb6f0d7455d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b81d1f86192240f493fdeb9b4badac8d, user_id=d464a830754344a5ba9f47380dc72801, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:38.269033 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd68aa5c-0652-4c19-93cf-bb6f0d7455d0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:38.281329 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1316/2627] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:36:38 2022] PUT /identity/v3/projects/b81d1f86192240f493fdeb9b4badac8d/users/d464a830754344a5ba9f47380dc72801/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:38.293558 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b873ae-16a1-4349-96e7-95bad4c2ca75 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:38.293907 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b873ae-16a1-4349-96e7-95bad4c2ca75 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:38.294298 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b873ae-16a1-4349-96e7-95bad4c2ca75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:38.327998 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-09b873ae-16a1-4349-96e7-95bad4c2ca75 None None] MFA Rules not processed for user `d464a830754344a5ba9f47380dc72801`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:38.329240 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-09b873ae-16a1-4349-96e7-95bad4c2ca75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:38.402726 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1312/2628] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:38 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:38.414021 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd68aa5c-0652-4c19-93cf-bb6f0d7455d0 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:38.415860 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-dd68aa5c-0652-4c19-93cf-bb6f0d7455d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:38.478652 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd68aa5c-0652-4c19-93cf-bb6f0d7455d0 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:38.482936 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3329f55-037c-44d9-a56b-5fe498ad7b29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:38.483546 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3329f55-037c-44d9-a56b-5fe498ad7b29 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:38.483817 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3329f55-037c-44d9-a56b-5fe498ad7b29 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:38.484088 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3329f55-037c-44d9-a56b-5fe498ad7b29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:38.494137 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3329f55-037c-44d9-a56b-5fe498ad7b29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:38.495605 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3329f55-037c-44d9-a56b-5fe498ad7b29 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:38.505165 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1317/2629] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:36:38 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:36:38.589732 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b873ae-16a1-4349-96e7-95bad4c2ca75 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:38.599517 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b873ae-16a1-4349-96e7-95bad4c2ca75 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:38.603075 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a839fa-e5a8-4593-a5e7-6416ae30bdb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:38.603682 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a839fa-e5a8-4593-a5e7-6416ae30bdb3 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:38.603960 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a839fa-e5a8-4593-a5e7-6416ae30bdb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:38.604228 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a839fa-e5a8-4593-a5e7-6416ae30bdb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:38.617066 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:38.663582 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a839fa-e5a8-4593-a5e7-6416ae30bdb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:38.665183 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a839fa-e5a8-4593-a5e7-6416ae30bdb3 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:38.675053 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1313/2630] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:36:38 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:36:38.844557 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3329f55-037c-44d9-a56b-5fe498ad7b29 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:38.846118 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e3329f55-037c-44d9-a56b-5fe498ad7b29 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:38.914350 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3329f55-037c-44d9-a56b-5fe498ad7b29 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:38.921110 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd700236-8bb9-4df6-a940-b93300221943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:38.921756 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd700236-8bb9-4df6-a940-b93300221943 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:38.922058 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd700236-8bb9-4df6-a940-b93300221943 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:38.922344 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd700236-8bb9-4df6-a940-b93300221943 admin admin] PATH_INFO: `/v3/users/afafb85b5e17441c845aaacecaca3ca8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:38.923692 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd700236-8bb9-4df6-a940-b93300221943 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afafb85b5e17441c845aaacecaca3ca8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:38.925620 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd700236-8bb9-4df6-a940-b93300221943 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:38.967617 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:38.976897 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-dd700236-8bb9-4df6-a940-b93300221943 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afafb85b5e17441c845aaacecaca3ca8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:38.977767 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-dd700236-8bb9-4df6-a940-b93300221943 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afafb85b5e17441c845aaacecaca3ca8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:38.978521 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-dd700236-8bb9-4df6-a940-b93300221943 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afafb85b5e17441c845aaacecaca3ca8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:38.986554 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-dd700236-8bb9-4df6-a940-b93300221943 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afafb85b5e17441c845aaacecaca3ca8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:38.993478 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-dd700236-8bb9-4df6-a940-b93300221943 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afafb85b5e17441c845aaacecaca3ca8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:39.010630 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1318/2631] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:36:38 2022] DELETE /identity/v3/users/afafb85b5e17441c845aaacecaca3ca8 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:39.015702 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cec5e5-6332-41d7-b782-9a735575076a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:39.015992 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cec5e5-6332-41d7-b782-9a735575076a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:39.016263 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cec5e5-6332-41d7-b782-9a735575076a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:39.036106 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-d3cec5e5-6332-41d7-b782-9a735575076a None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:39.037317 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d3cec5e5-6332-41d7-b782-9a735575076a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:39.097784 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1314/2632] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:39.110527 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd700236-8bb9-4df6-a940-b93300221943 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:39.113008 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-dd700236-8bb9-4df6-a940-b93300221943 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:39.177253 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd700236-8bb9-4df6-a940-b93300221943 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:39.181288 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293892a6-752a-4007-924e-38141a66fab2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:39.181912 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293892a6-752a-4007-924e-38141a66fab2 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:39.182187 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293892a6-752a-4007-924e-38141a66fab2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:39.182460 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293892a6-752a-4007-924e-38141a66fab2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:39.192070 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293892a6-752a-4007-924e-38141a66fab2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:39.193510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293892a6-752a-4007-924e-38141a66fab2 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:39.203144 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1319/2633] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:36:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:36:39.480930 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cec5e5-6332-41d7-b782-9a735575076a None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:39.482545 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d3cec5e5-6332-41d7-b782-9a735575076a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:39.495026 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cec5e5-6332-41d7-b782-9a735575076a None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:39.503109 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c2ec5b-b477-40ac-87bb-ca2fe91b56fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:39.504630 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2ec5b-b477-40ac-87bb-ca2fe91b56fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:39.505236 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2ec5b-b477-40ac-87bb-ca2fe91b56fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:39.505515 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2ec5b-b477-40ac-87bb-ca2fe91b56fa admin admin] PATH_INFO: `/v3/projects/f5c7be2e565d44cebf1aba36f7b3720c` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:39.506726 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c2ec5b-b477-40ac-87bb-ca2fe91b56fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5c7be2e565d44cebf1aba36f7b3720c)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:39.508726 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c2ec5b-b477-40ac-87bb-ca2fe91b56fa admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:39.560375 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e7c2ec5b-b477-40ac-87bb-ca2fe91b56fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5c7be2e565d44cebf1aba36f7b3720c', 'request_id': 'req-e7c2ec5b-b477-40ac-87bb-ca2fe91b56fa'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:39.565666 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e7c2ec5b-b477-40ac-87bb-ca2fe91b56fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5c7be2e565d44cebf1aba36f7b3720c', 'request_id': 'req-e7c2ec5b-b477-40ac-87bb-ca2fe91b56fa'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:39.576049 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e7c2ec5b-b477-40ac-87bb-ca2fe91b56fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5c7be2e565d44cebf1aba36f7b3720c', 'request_id': 'req-e7c2ec5b-b477-40ac-87bb-ca2fe91b56fa'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:39.587884 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e7c2ec5b-b477-40ac-87bb-ca2fe91b56fa admin admin] The token cache is being invalidate because project f5c7be2e565d44cebf1aba36f7b3720c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:36:39.588196 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e7c2ec5b-b477-40ac-87bb-ca2fe91b56fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:39.588828 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e7c2ec5b-b477-40ac-87bb-ca2fe91b56fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:39.590450 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1315/2634] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:36:39 2022] DELETE /identity/v3/projects/f5c7be2e565d44cebf1aba36f7b3720c => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:39.595628 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d86e7-5d15-4561-b202-a9deb6a3f487 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:39.596015 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d86e7-5d15-4561-b202-a9deb6a3f487 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:39.596170 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d86e7-5d15-4561-b202-a9deb6a3f487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:39.616234 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-cf3d86e7-5d15-4561-b202-a9deb6a3f487 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:39.617357 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cf3d86e7-5d15-4561-b202-a9deb6a3f487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:39.692589 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1320/2635] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:39.700532 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b4fead-57b6-44a9-8f42-e98a95262e58 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:39.700821 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b4fead-57b6-44a9-8f42-e98a95262e58 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:39.701083 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b4fead-57b6-44a9-8f42-e98a95262e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:39.722120 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-33b4fead-57b6-44a9-8f42-e98a95262e58 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:39.723354 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-33b4fead-57b6-44a9-8f42-e98a95262e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:39.732563 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1316/2636] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:39.736987 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3d86e7-5d15-4561-b202-a9deb6a3f487 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:39.748098 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3d86e7-5d15-4561-b202-a9deb6a3f487 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:39.753615 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d24f10d-ab8c-4fbb-b113-33e3c3d17c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:39.754239 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d24f10d-ab8c-4fbb-b113-33e3c3d17c61 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:39.754517 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d24f10d-ab8c-4fbb-b113-33e3c3d17c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:39.754788 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d24f10d-ab8c-4fbb-b113-33e3c3d17c61 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:39.755523 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d24f10d-ab8c-4fbb-b113-33e3c3d17c61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:36:39.755826 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d24f10d-ab8c-4fbb-b113-33e3c3d17c61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:39.757437 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d24f10d-ab8c-4fbb-b113-33e3c3d17c61 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:39.770963 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1321/2637] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:36:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:36:39.775317 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b4fead-57b6-44a9-8f42-e98a95262e58 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:39.784699 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b4fead-57b6-44a9-8f42-e98a95262e58 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:39.790113 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b44592f-982a-4781-8151-2bc12227fb74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:39.790732 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44592f-982a-4781-8151-2bc12227fb74 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:39.791045 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44592f-982a-4781-8151-2bc12227fb74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:39.791315 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44592f-982a-4781-8151-2bc12227fb74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:39.791984 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b44592f-982a-4781-8151-2bc12227fb74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:39.794049 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b44592f-982a-4781-8151-2bc12227fb74 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:39.826615 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1317/2638] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:36:39 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:36:39.830774 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d24f10d-ab8c-4fbb-b113-33e3c3d17c61 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:39.841532 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d24f10d-ab8c-4fbb-b113-33e3c3d17c61 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:39.891874 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:39.908884 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed8e767-2173-45f7-af2d-edc9933d2428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:39.909585 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8e767-2173-45f7-af2d-edc9933d2428 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:39.909782 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8e767-2173-45f7-af2d-edc9933d2428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:39.910054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8e767-2173-45f7-af2d-edc9933d2428 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:39.910723 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed8e767-2173-45f7-af2d-edc9933d2428 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:39.912736 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed8e767-2173-45f7-af2d-edc9933d2428 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:39.938059 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1322/2639] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:36:39 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:36:39.942285 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b44592f-982a-4781-8151-2bc12227fb74 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:39.953313 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b44592f-982a-4781-8151-2bc12227fb74 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:39.958092 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3092b7-5fc5-41c0-ad2b-f0d994e5298f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:39.958701 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3092b7-5fc5-41c0-ad2b-f0d994e5298f admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:39.959015 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3092b7-5fc5-41c0-ad2b-f0d994e5298f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:39.959287 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3092b7-5fc5-41c0-ad2b-f0d994e5298f admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:39.959983 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3092b7-5fc5-41c0-ad2b-f0d994e5298f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:39.961571 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3092b7-5fc5-41c0-ad2b-f0d994e5298f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:39.967604 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:39.973108 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1318/2640] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:36:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:36:39.978688 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed8e767-2173-45f7-af2d-edc9933d2428 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:39.989546 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed8e767-2173-45f7-af2d-edc9933d2428 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:39.994951 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732f9fd3-0efc-49fb-9500-4a4bc76f88e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:39.995610 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f9fd3-0efc-49fb-9500-4a4bc76f88e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:39.995906 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f9fd3-0efc-49fb-9500-4a4bc76f88e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:39.996149 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f9fd3-0efc-49fb-9500-4a4bc76f88e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:39.996872 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732f9fd3-0efc-49fb-9500-4a4bc76f88e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:39.998490 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732f9fd3-0efc-49fb-9500-4a4bc76f88e5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:40.010617 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1323/2641] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:36:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:36:40.016332 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3092b7-5fc5-41c0-ad2b-f0d994e5298f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:40.027699 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3092b7-5fc5-41c0-ad2b-f0d994e5298f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:40.033048 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c126081-0dcd-41e7-9838-e25d9ede946f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:40.033718 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c126081-0dcd-41e7-9838-e25d9ede946f admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:40.034121 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c126081-0dcd-41e7-9838-e25d9ede946f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:40.034400 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c126081-0dcd-41e7-9838-e25d9ede946f admin admin] PATH_INFO: `/v3/projects/ec4852dfe48a49f0ba3548bf2bbfb9a4/users/0c7029b92f514b689235f130e7da5809/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:40.049929 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c126081-0dcd-41e7-9838-e25d9ede946f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec4852dfe48a49f0ba3548bf2bbfb9a4, user_id=0c7029b92f514b689235f130e7da5809, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:40.052653 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c126081-0dcd-41e7-9838-e25d9ede946f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:40.074392 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1319/2642] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:36:40 2022] PUT /identity/v3/projects/ec4852dfe48a49f0ba3548bf2bbfb9a4/users/0c7029b92f514b689235f130e7da5809/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:40.080105 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d07bff-bbb7-4176-8cb1-f15456ff1461 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:40.080873 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d07bff-bbb7-4176-8cb1-f15456ff1461 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:40.081635 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d07bff-bbb7-4176-8cb1-f15456ff1461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:40.109370 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-94d07bff-bbb7-4176-8cb1-f15456ff1461 None None] MFA Rules not processed for user `0c7029b92f514b689235f130e7da5809`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:40.110542 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-94d07bff-bbb7-4176-8cb1-f15456ff1461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:40.213586 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1324/2643] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:40.225740 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c126081-0dcd-41e7-9838-e25d9ede946f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:40.228313 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4c126081-0dcd-41e7-9838-e25d9ede946f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:40.297340 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c126081-0dcd-41e7-9838-e25d9ede946f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:40.301435 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c985a80f-8868-4aae-a6c9-76d3c86f9117 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:40.302060 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985a80f-8868-4aae-a6c9-76d3c86f9117 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:40.302343 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985a80f-8868-4aae-a6c9-76d3c86f9117 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:40.302616 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985a80f-8868-4aae-a6c9-76d3c86f9117 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:40.312727 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c985a80f-8868-4aae-a6c9-76d3c86f9117 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:40.312937 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c985a80f-8868-4aae-a6c9-76d3c86f9117 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:40.322572 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1320/2644] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:36:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:36:47.149547 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d07bff-bbb7-4176-8cb1-f15456ff1461 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:47.216291 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d07bff-bbb7-4176-8cb1-f15456ff1461 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:47.221362 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b742294-8907-4539-852c-ebfc43ce7481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:47.222022 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b742294-8907-4539-852c-ebfc43ce7481 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:47.222300 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b742294-8907-4539-852c-ebfc43ce7481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:47.222572 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b742294-8907-4539-852c-ebfc43ce7481 admin admin] PATH_INFO: `/v3/users/0c7029b92f514b689235f130e7da5809` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:47.223845 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b742294-8907-4539-852c-ebfc43ce7481 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c7029b92f514b689235f130e7da5809)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:47.225815 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b742294-8907-4539-852c-ebfc43ce7481 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:47.280849 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6b742294-8907-4539-852c-ebfc43ce7481 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c7029b92f514b689235f130e7da5809'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:47.281531 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6b742294-8907-4539-852c-ebfc43ce7481 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c7029b92f514b689235f130e7da5809'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:47.282325 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6b742294-8907-4539-852c-ebfc43ce7481 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c7029b92f514b689235f130e7da5809'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:47.291491 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6b742294-8907-4539-852c-ebfc43ce7481 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c7029b92f514b689235f130e7da5809'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:47.292949 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:47.300586 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6b742294-8907-4539-852c-ebfc43ce7481 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c7029b92f514b689235f130e7da5809'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:47.320698 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1325/2645] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:36:47 2022] DELETE /identity/v3/users/0c7029b92f514b689235f130e7da5809 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:47.325384 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9c93f-3a51-43fd-a8ae-0aab00744a88 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:47.325953 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9c93f-3a51-43fd-a8ae-0aab00744a88 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:47.325953 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9c93f-3a51-43fd-a8ae-0aab00744a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:47.346629 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-5ec9c93f-3a51-43fd-a8ae-0aab00744a88 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:47.347954 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5ec9c93f-3a51-43fd-a8ae-0aab00744a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:47.412548 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1321/2646] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:47.422591 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b742294-8907-4539-852c-ebfc43ce7481 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:47.424159 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6b742294-8907-4539-852c-ebfc43ce7481 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:47.487272 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b742294-8907-4539-852c-ebfc43ce7481 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:47.490932 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b26fd-df0c-4013-a6fc-f971da949c4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:47.491558 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b26fd-df0c-4013-a6fc-f971da949c4b service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:47.491838 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b26fd-df0c-4013-a6fc-f971da949c4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:47.492110 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b26fd-df0c-4013-a6fc-f971da949c4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:47.502185 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37b26fd-df0c-4013-a6fc-f971da949c4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:47.504516 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37b26fd-df0c-4013-a6fc-f971da949c4b service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:47.514779 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1326/2647] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:36:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:36:47.795747 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec9c93f-3a51-43fd-a8ae-0aab00744a88 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:47.797398 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5ec9c93f-3a51-43fd-a8ae-0aab00744a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:47.810636 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec9c93f-3a51-43fd-a8ae-0aab00744a88 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:47.816516 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f0751-96fe-4170-9082-ac9449b38273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:47.817584 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f0751-96fe-4170-9082-ac9449b38273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:47.817584 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f0751-96fe-4170-9082-ac9449b38273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:47.817803 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f0751-96fe-4170-9082-ac9449b38273 admin admin] PATH_INFO: `/v3/projects/ec4852dfe48a49f0ba3548bf2bbfb9a4` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:47.819049 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f0751-96fe-4170-9082-ac9449b38273 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec4852dfe48a49f0ba3548bf2bbfb9a4)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:47.821175 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f0751-96fe-4170-9082-ac9449b38273 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:47.873715 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4e7f0751-96fe-4170-9082-ac9449b38273 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec4852dfe48a49f0ba3548bf2bbfb9a4', 'request_id': 'req-4e7f0751-96fe-4170-9082-ac9449b38273'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:47.879575 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4e7f0751-96fe-4170-9082-ac9449b38273 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec4852dfe48a49f0ba3548bf2bbfb9a4', 'request_id': 'req-4e7f0751-96fe-4170-9082-ac9449b38273'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:47.889189 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4e7f0751-96fe-4170-9082-ac9449b38273 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec4852dfe48a49f0ba3548bf2bbfb9a4', 'request_id': 'req-4e7f0751-96fe-4170-9082-ac9449b38273'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:47.900716 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4e7f0751-96fe-4170-9082-ac9449b38273 admin admin] The token cache is being invalidate because project ec4852dfe48a49f0ba3548bf2bbfb9a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:36:47.901014 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4e7f0751-96fe-4170-9082-ac9449b38273 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:47.901801 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4e7f0751-96fe-4170-9082-ac9449b38273 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:47.903529 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1322/2648] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:36:47 2022] DELETE /identity/v3/projects/ec4852dfe48a49f0ba3548bf2bbfb9a4 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:49.624028 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b26fd-df0c-4013-a6fc-f971da949c4b service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:49.625509 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f37b26fd-df0c-4013-a6fc-f971da949c4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:49.637731 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b26fd-df0c-4013-a6fc-f971da949c4b service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:49.689856 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb2a431-245e-46e1-b63e-9bc2e26ee794 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:49.690468 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2a431-245e-46e1-b63e-9bc2e26ee794 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:49.690747 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2a431-245e-46e1-b63e-9bc2e26ee794 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:49.691049 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2a431-245e-46e1-b63e-9bc2e26ee794 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:49.693042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cbb2a431-245e-46e1-b63e-9bc2e26ee794 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:49.764559 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb2a431-245e-46e1-b63e-9bc2e26ee794 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:49.766003 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb2a431-245e-46e1-b63e-9bc2e26ee794 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:49.776285 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1327/2649] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:36:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:36:50.229187 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f0751-96fe-4170-9082-ac9449b38273 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:50.230747 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4e7f0751-96fe-4170-9082-ac9449b38273 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:50.239071 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:50.244371 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f0751-96fe-4170-9082-ac9449b38273 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:50.251964 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0624f3a-6e66-4438-9184-e07482e9b1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:50.252981 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0624f3a-6e66-4438-9184-e07482e9b1bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:50.253268 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0624f3a-6e66-4438-9184-e07482e9b1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:50.253802 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0624f3a-6e66-4438-9184-e07482e9b1bf admin admin] PATH_INFO: `/v3/users/5e67ca02d15e4a20b972630b7fd02335` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:50.255667 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0624f3a-6e66-4438-9184-e07482e9b1bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e67ca02d15e4a20b972630b7fd02335)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:50.257622 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0624f3a-6e66-4438-9184-e07482e9b1bf admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:50.311165 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e0624f3a-6e66-4438-9184-e07482e9b1bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e67ca02d15e4a20b972630b7fd02335'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:50.320383 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e0624f3a-6e66-4438-9184-e07482e9b1bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e67ca02d15e4a20b972630b7fd02335'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:50.321177 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e0624f3a-6e66-4438-9184-e07482e9b1bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e67ca02d15e4a20b972630b7fd02335'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:50.333980 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e0624f3a-6e66-4438-9184-e07482e9b1bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e67ca02d15e4a20b972630b7fd02335'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:50.334671 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e0624f3a-6e66-4438-9184-e07482e9b1bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e67ca02d15e4a20b972630b7fd02335'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:50.347211 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1323/2650] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:36:50 2022] DELETE /identity/v3/users/5e67ca02d15e4a20b972630b7fd02335 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:50.351866 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b405c-1a41-4d5b-b6d4-77dbc4c539ea None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:50.352152 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b405c-1a41-4d5b-b6d4-77dbc4c539ea None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:50.352425 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b405c-1a41-4d5b-b6d4-77dbc4c539ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:50.371940 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-212b405c-1a41-4d5b-b6d4-77dbc4c539ea None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:50.373343 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-212b405c-1a41-4d5b-b6d4-77dbc4c539ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:50.393893 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:50.438262 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1328/2651] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:50.449003 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0624f3a-6e66-4438-9184-e07482e9b1bf admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:50.450490 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e0624f3a-6e66-4438-9184-e07482e9b1bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:50.518806 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0624f3a-6e66-4438-9184-e07482e9b1bf admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:50.523018 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d9fb35-65eb-44e1-8bb7-a4cdd6848260 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:50.523649 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d9fb35-65eb-44e1-8bb7-a4cdd6848260 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:50.523929 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d9fb35-65eb-44e1-8bb7-a4cdd6848260 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:50.524173 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d9fb35-65eb-44e1-8bb7-a4cdd6848260 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:50.534105 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d9fb35-65eb-44e1-8bb7-a4cdd6848260 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:50.535733 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d9fb35-65eb-44e1-8bb7-a4cdd6848260 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:50.546313 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1324/2652] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:36:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:36:50.840005 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b405c-1a41-4d5b-b6d4-77dbc4c539ea None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:50.841573 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-212b405c-1a41-4d5b-b6d4-77dbc4c539ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:50.857583 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b405c-1a41-4d5b-b6d4-77dbc4c539ea None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:50.863583 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007bb926-b99d-4f1b-bb4d-04d9308f6698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:50.864278 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007bb926-b99d-4f1b-bb4d-04d9308f6698 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:50.864554 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007bb926-b99d-4f1b-bb4d-04d9308f6698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:50.864824 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007bb926-b99d-4f1b-bb4d-04d9308f6698 admin admin] PATH_INFO: `/v3/projects/f001bcce76084f89a4efe99550e5bc80` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:50.867479 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007bb926-b99d-4f1b-bb4d-04d9308f6698 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f001bcce76084f89a4efe99550e5bc80)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:50.870265 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007bb926-b99d-4f1b-bb4d-04d9308f6698 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:50.923469 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-007bb926-b99d-4f1b-bb4d-04d9308f6698 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f001bcce76084f89a4efe99550e5bc80', 'request_id': 'req-007bb926-b99d-4f1b-bb4d-04d9308f6698'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:50.933214 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-007bb926-b99d-4f1b-bb4d-04d9308f6698 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f001bcce76084f89a4efe99550e5bc80', 'request_id': 'req-007bb926-b99d-4f1b-bb4d-04d9308f6698'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:50.940380 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-007bb926-b99d-4f1b-bb4d-04d9308f6698 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f001bcce76084f89a4efe99550e5bc80', 'request_id': 'req-007bb926-b99d-4f1b-bb4d-04d9308f6698'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:50.951509 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-007bb926-b99d-4f1b-bb4d-04d9308f6698 admin admin] The token cache is being invalidate because project f001bcce76084f89a4efe99550e5bc80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:36:50.951798 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-007bb926-b99d-4f1b-bb4d-04d9308f6698 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:50.952450 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-007bb926-b99d-4f1b-bb4d-04d9308f6698 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:50.953984 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1329/2653] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:36:50 2022] DELETE /identity/v3/projects/f001bcce76084f89a4efe99550e5bc80 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:50.958119 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d9fb35-65eb-44e1-8bb7-a4cdd6848260 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:50.968949 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d9fb35-65eb-44e1-8bb7-a4cdd6848260 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:51.033457 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3662636-53ca-48ca-b84b-eb582dc81fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:51.034175 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3662636-53ca-48ca-b84b-eb582dc81fba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:51.034456 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3662636-53ca-48ca-b84b-eb582dc81fba admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:51.034722 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3662636-53ca-48ca-b84b-eb582dc81fba admin admin] PATH_INFO: `/v3/users/d464a830754344a5ba9f47380dc72801` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:51.035968 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3662636-53ca-48ca-b84b-eb582dc81fba admin admin] RBAC: Authorizing `identity:delete_user(user_id=d464a830754344a5ba9f47380dc72801)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:51.037904 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3662636-53ca-48ca-b84b-eb582dc81fba admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:51.058892 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:51.088844 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c3662636-53ca-48ca-b84b-eb582dc81fba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd464a830754344a5ba9f47380dc72801'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:51.097432 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c3662636-53ca-48ca-b84b-eb582dc81fba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd464a830754344a5ba9f47380dc72801'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:51.098512 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c3662636-53ca-48ca-b84b-eb582dc81fba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd464a830754344a5ba9f47380dc72801'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:51.112607 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c3662636-53ca-48ca-b84b-eb582dc81fba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd464a830754344a5ba9f47380dc72801'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:51.113971 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c3662636-53ca-48ca-b84b-eb582dc81fba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd464a830754344a5ba9f47380dc72801'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:51.127802 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1325/2654] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:36:50 2022] DELETE /identity/v3/users/d464a830754344a5ba9f47380dc72801 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:36:51.515428 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007bb926-b99d-4f1b-bb4d-04d9308f6698 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:51.516875 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-007bb926-b99d-4f1b-bb4d-04d9308f6698 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:51.530127 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007bb926-b99d-4f1b-bb4d-04d9308f6698 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:51.597984 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a525c3e-2ebd-42eb-93ce-e72d917eb393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:51.598553 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a525c3e-2ebd-42eb-93ce-e72d917eb393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:51.598835 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a525c3e-2ebd-42eb-93ce-e72d917eb393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:51.599143 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a525c3e-2ebd-42eb-93ce-e72d917eb393 admin admin] PATH_INFO: `/v3/projects/b81d1f86192240f493fdeb9b4badac8d` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:51.600343 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a525c3e-2ebd-42eb-93ce-e72d917eb393 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b81d1f86192240f493fdeb9b4badac8d)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:51.603449 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a525c3e-2ebd-42eb-93ce-e72d917eb393 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:51.677662 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6a525c3e-2ebd-42eb-93ce-e72d917eb393 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b81d1f86192240f493fdeb9b4badac8d', 'request_id': 'req-6a525c3e-2ebd-42eb-93ce-e72d917eb393'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:51.680042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:51.693262 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6a525c3e-2ebd-42eb-93ce-e72d917eb393 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b81d1f86192240f493fdeb9b4badac8d', 'request_id': 'req-6a525c3e-2ebd-42eb-93ce-e72d917eb393'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:51.704514 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6a525c3e-2ebd-42eb-93ce-e72d917eb393 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b81d1f86192240f493fdeb9b4badac8d', 'request_id': 'req-6a525c3e-2ebd-42eb-93ce-e72d917eb393'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:51.720209 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6a525c3e-2ebd-42eb-93ce-e72d917eb393 admin admin] The token cache is being invalidate because project b81d1f86192240f493fdeb9b4badac8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:36:51.720506 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6a525c3e-2ebd-42eb-93ce-e72d917eb393 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:51.721162 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6a525c3e-2ebd-42eb-93ce-e72d917eb393 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:51.726147 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1330/2655] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:36:51 2022] DELETE /identity/v3/projects/b81d1f86192240f493fdeb9b4badac8d => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:51.728137 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1fe2fd-c643-4e8e-9e50-42108bfdb084 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:51.728614 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1fe2fd-c643-4e8e-9e50-42108bfdb084 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:51.728701 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1fe2fd-c643-4e8e-9e50-42108bfdb084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:51.751645 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-5d1fe2fd-c643-4e8e-9e50-42108bfdb084 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:51.753005 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5d1fe2fd-c643-4e8e-9e50-42108bfdb084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:51.824859 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1326/2656] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:51.834142 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222af6f5-4653-40ce-9865-d085acd0d5d7 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:51.835555 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222af6f5-4653-40ce-9865-d085acd0d5d7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:51.837032 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222af6f5-4653-40ce-9865-d085acd0d5d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:51.858148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-222af6f5-4653-40ce-9865-d085acd0d5d7 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:51.859517 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-222af6f5-4653-40ce-9865-d085acd0d5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:51.867940 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1331/2657] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:51.872405 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1fe2fd-c643-4e8e-9e50-42108bfdb084 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:51.882480 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1fe2fd-c643-4e8e-9e50-42108bfdb084 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:51.887324 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09128e55-288f-40c5-b932-f7132baa0131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:51.887894 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09128e55-288f-40c5-b932-f7132baa0131 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:51.888172 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09128e55-288f-40c5-b932-f7132baa0131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:51.888445 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09128e55-288f-40c5-b932-f7132baa0131 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:51.889601 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09128e55-288f-40c5-b932-f7132baa0131 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:36:51.889800 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09128e55-288f-40c5-b932-f7132baa0131 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:51.891662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09128e55-288f-40c5-b932-f7132baa0131 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:51.906876 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1327/2658] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:36:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:36:51.909385 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222af6f5-4653-40ce-9865-d085acd0d5d7 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:51.917927 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222af6f5-4653-40ce-9865-d085acd0d5d7 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:51.923064 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab04256-9d5a-4f42-9b67-1e4fdf2adeff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:51.923903 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab04256-9d5a-4f42-9b67-1e4fdf2adeff admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:51.924531 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab04256-9d5a-4f42-9b67-1e4fdf2adeff admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:51.924620 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab04256-9d5a-4f42-9b67-1e4fdf2adeff admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:51.925475 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab04256-9d5a-4f42-9b67-1e4fdf2adeff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:51.927648 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab04256-9d5a-4f42-9b67-1e4fdf2adeff admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:51.967794 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1332/2659] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:36:51 2022] POST /identity/v3/projects => generated 387 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:36:51.973492 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09128e55-288f-40c5-b932-f7132baa0131 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:51.983844 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09128e55-288f-40c5-b932-f7132baa0131 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:52.063672 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7380ed71-fe9f-4f81-a10f-93dbdab1ded4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:52.064314 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7380ed71-fe9f-4f81-a10f-93dbdab1ded4 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:52.064601 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7380ed71-fe9f-4f81-a10f-93dbdab1ded4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:52.064877 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7380ed71-fe9f-4f81-a10f-93dbdab1ded4 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:52.065569 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7380ed71-fe9f-4f81-a10f-93dbdab1ded4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:52.068484 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7380ed71-fe9f-4f81-a10f-93dbdab1ded4 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:52.094213 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1328/2660] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:36:51 2022] POST /identity/v3/users => generated 294 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:36:52.098418 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab04256-9d5a-4f42-9b67-1e4fdf2adeff admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:52.109990 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab04256-9d5a-4f42-9b67-1e4fdf2adeff admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:52.114502 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073bc4bc-98e0-46ed-a181-6e433d3c6b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:52.115155 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073bc4bc-98e0-46ed-a181-6e433d3c6b18 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:52.115439 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073bc4bc-98e0-46ed-a181-6e433d3c6b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:52.115715 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073bc4bc-98e0-46ed-a181-6e433d3c6b18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:52.116410 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073bc4bc-98e0-46ed-a181-6e433d3c6b18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:52.118049 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073bc4bc-98e0-46ed-a181-6e433d3c6b18 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:52.131691 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1333/2661] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:36:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 10:36:52.132965 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7380ed71-fe9f-4f81-a10f-93dbdab1ded4 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:52.144282 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7380ed71-fe9f-4f81-a10f-93dbdab1ded4 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:52.149728 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe70b48-1c99-4464-8ba6-8d3681ade17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:52.150357 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe70b48-1c99-4464-8ba6-8d3681ade17b admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:52.150630 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe70b48-1c99-4464-8ba6-8d3681ade17b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:52.150881 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe70b48-1c99-4464-8ba6-8d3681ade17b admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:52.151611 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe70b48-1c99-4464-8ba6-8d3681ade17b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:52.153581 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe70b48-1c99-4464-8ba6-8d3681ade17b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:52.172116 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1329/2662] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:36:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:36:52.173500 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073bc4bc-98e0-46ed-a181-6e433d3c6b18 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:52.184607 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073bc4bc-98e0-46ed-a181-6e433d3c6b18 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:52.189762 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9358ce94-815c-46ce-81cd-009dd38062b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:52.190437 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358ce94-815c-46ce-81cd-009dd38062b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:52.190831 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358ce94-815c-46ce-81cd-009dd38062b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:52.191146 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358ce94-815c-46ce-81cd-009dd38062b2 admin admin] PATH_INFO: `/v3/projects/72f8d9c76a5e4d1682113aaca1652d74/users/f72c79adfb294d11b42f5b0465e3014d/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:52.208134 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9358ce94-815c-46ce-81cd-009dd38062b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72f8d9c76a5e4d1682113aaca1652d74, user_id=f72c79adfb294d11b42f5b0465e3014d, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:52.211549 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9358ce94-815c-46ce-81cd-009dd38062b2 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:52.230236 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1334/2663] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:36:52 2022] PUT /identity/v3/projects/72f8d9c76a5e4d1682113aaca1652d74/users/f72c79adfb294d11b42f5b0465e3014d/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:52.239610 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a2805-60fb-47d5-aeeb-693d10703a3f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:52.239610 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a2805-60fb-47d5-aeeb-693d10703a3f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:52.239902 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a2805-60fb-47d5-aeeb-693d10703a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:52.252380 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:52.282005 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a76a2805-60fb-47d5-aeeb-693d10703a3f None None] MFA Rules not processed for user `f72c79adfb294d11b42f5b0465e3014d`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:52.283418 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a76a2805-60fb-47d5-aeeb-693d10703a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:52.358123 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1330/2664] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:52 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:52.362649 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9358ce94-815c-46ce-81cd-009dd38062b2 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:52.435495 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9358ce94-815c-46ce-81cd-009dd38062b2 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:52.440579 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a078c1c-5bf6-4213-9069-74b36dcf354d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:52.441235 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a078c1c-5bf6-4213-9069-74b36dcf354d admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:52.441516 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a078c1c-5bf6-4213-9069-74b36dcf354d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:52.441837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a078c1c-5bf6-4213-9069-74b36dcf354d admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:52.442467 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a078c1c-5bf6-4213-9069-74b36dcf354d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:52.445148 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a078c1c-5bf6-4213-9069-74b36dcf354d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:52.475601 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1335/2665] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:36:52 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:36:52.479865 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a2805-60fb-47d5-aeeb-693d10703a3f None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:52.502927 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a2805-60fb-47d5-aeeb-693d10703a3f None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:52.580214 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477af6b-b1ab-4dc4-b26c-a89c99e16478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:52.580853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477af6b-b1ab-4dc4-b26c-a89c99e16478 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:52.581129 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477af6b-b1ab-4dc4-b26c-a89c99e16478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:52.581395 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477af6b-b1ab-4dc4-b26c-a89c99e16478 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:52.582102 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8477af6b-b1ab-4dc4-b26c-a89c99e16478 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:52.584066 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8477af6b-b1ab-4dc4-b26c-a89c99e16478 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:52.610920 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1331/2666] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:36:52 2022] POST /identity/v3/users => generated 299 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:36:52.615202 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a078c1c-5bf6-4213-9069-74b36dcf354d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:52.626268 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a078c1c-5bf6-4213-9069-74b36dcf354d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:52.631837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded1fe74-597c-413e-aa30-a1125a6b6213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:52.632552 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1fe74-597c-413e-aa30-a1125a6b6213 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:52.632835 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1fe74-597c-413e-aa30-a1125a6b6213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:52.633109 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1fe74-597c-413e-aa30-a1125a6b6213 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:52.633809 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded1fe74-597c-413e-aa30-a1125a6b6213 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:52.635459 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded1fe74-597c-413e-aa30-a1125a6b6213 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:52.651066 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1336/2667] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:36:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:36:52.654001 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477af6b-b1ab-4dc4-b26c-a89c99e16478 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:52.671446 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477af6b-b1ab-4dc4-b26c-a89c99e16478 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:52.682615 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1d760f-b18f-4475-89ba-cb44e75985e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:52.683929 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d760f-b18f-4475-89ba-cb44e75985e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:52.685098 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d760f-b18f-4475-89ba-cb44e75985e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:52.685485 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d760f-b18f-4475-89ba-cb44e75985e9 admin admin] PATH_INFO: `/v3/projects/24987d43a2784320b9746268c2193144/users/3040e9c551434b53b548e4645b948ec5/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:52.707768 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1d760f-b18f-4475-89ba-cb44e75985e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24987d43a2784320b9746268c2193144, user_id=3040e9c551434b53b548e4645b948ec5, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:52.710554 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1d760f-b18f-4475-89ba-cb44e75985e9 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:52.731041 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1332/2668] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:36:52 2022] PUT /identity/v3/projects/24987d43a2784320b9746268c2193144/users/3040e9c551434b53b548e4645b948ec5/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:52.731718 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded1fe74-597c-413e-aa30-a1125a6b6213 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:52.747962 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded1fe74-597c-413e-aa30-a1125a6b6213 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:52.801759 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:52.824255 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bba011-27bf-44c6-b24f-31e250faa2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:52.826476 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bba011-27bf-44c6-b24f-31e250faa2f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:52.826862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bba011-27bf-44c6-b24f-31e250faa2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:52.827263 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bba011-27bf-44c6-b24f-31e250faa2f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:52.828404 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bba011-27bf-44c6-b24f-31e250faa2f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:52.830774 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bba011-27bf-44c6-b24f-31e250faa2f1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:52.854805 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1337/2669] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:36:52 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 10:36:52.859436 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1d760f-b18f-4475-89ba-cb44e75985e9 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:52.870314 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1d760f-b18f-4475-89ba-cb44e75985e9 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:52.875944 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c8e885-c3c1-4daa-9604-d4b112a4a586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:52.876636 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c8e885-c3c1-4daa-9604-d4b112a4a586 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:52.880329 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c8e885-c3c1-4daa-9604-d4b112a4a586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:52.880614 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c8e885-c3c1-4daa-9604-d4b112a4a586 admin admin] PATH_INFO: `/v3/projects/24987d43a2784320b9746268c2193144/users/3040e9c551434b53b548e4645b948ec5/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:52.885739 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c8e885-c3c1-4daa-9604-d4b112a4a586 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24987d43a2784320b9746268c2193144, user_id=3040e9c551434b53b548e4645b948ec5, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:52.889601 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c8e885-c3c1-4daa-9604-d4b112a4a586 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:52.901003 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1333/2670] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:36:52 2022] PUT /identity/v3/projects/24987d43a2784320b9746268c2193144/users/3040e9c551434b53b548e4645b948ec5/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:52.912775 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73113bb8-ccaa-4b14-b087-ec3d1a5b2f21 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:52.913256 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73113bb8-ccaa-4b14-b087-ec3d1a5b2f21 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:52.913256 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73113bb8-ccaa-4b14-b087-ec3d1a5b2f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:52.944097 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-73113bb8-ccaa-4b14-b087-ec3d1a5b2f21 None None] MFA Rules not processed for user `3040e9c551434b53b548e4645b948ec5`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:52.945873 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-73113bb8-ccaa-4b14-b087-ec3d1a5b2f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:53.074008 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1338/2671] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:52 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:53.093895 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c8e885-c3c1-4daa-9604-d4b112a4a586 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:53.096835 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-11c8e885-c3c1-4daa-9604-d4b112a4a586 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:53.099411 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73113bb8-ccaa-4b14-b087-ec3d1a5b2f21 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:53.101050 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-73113bb8-ccaa-4b14-b087-ec3d1a5b2f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:53.150376 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:53.163950 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c8e885-c3c1-4daa-9604-d4b112a4a586 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:53.168538 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8bf042-1ab0-40c2-acbc-ca46a062bcd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:53.169168 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8bf042-1ab0-40c2-acbc-ca46a062bcd0 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:53.169446 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8bf042-1ab0-40c2-acbc-ca46a062bcd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:53.169718 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8bf042-1ab0-40c2-acbc-ca46a062bcd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:53.178972 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73113bb8-ccaa-4b14-b087-ec3d1a5b2f21 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:53.179139 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8bf042-1ab0-40c2-acbc-ca46a062bcd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:53.180456 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8bf042-1ab0-40c2-acbc-ca46a062bcd0 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:53.183735 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1f79f-beca-4070-ba94-2b3367a98b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:53.185186 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1f79f-beca-4070-ba94-2b3367a98b8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:53.185466 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1f79f-beca-4070-ba94-2b3367a98b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:53.185741 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1f79f-beca-4070-ba94-2b3367a98b8f admin admin] PATH_INFO: `/v3/users/97f671de92d240ae85ca4893d1ff04e5` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:53.190562 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1334/2672] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:36:53 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:36:53.202883 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be1f79f-beca-4070-ba94-2b3367a98b8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=97f671de92d240ae85ca4893d1ff04e5)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:53.204846 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be1f79f-beca-4070-ba94-2b3367a98b8f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:53.260942 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3be1f79f-beca-4070-ba94-2b3367a98b8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97f671de92d240ae85ca4893d1ff04e5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:53.261644 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3be1f79f-beca-4070-ba94-2b3367a98b8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97f671de92d240ae85ca4893d1ff04e5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:53.262570 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3be1f79f-beca-4070-ba94-2b3367a98b8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97f671de92d240ae85ca4893d1ff04e5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:53.263428 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8bf042-1ab0-40c2-acbc-ca46a062bcd0 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:53.273679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3be1f79f-beca-4070-ba94-2b3367a98b8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97f671de92d240ae85ca4893d1ff04e5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:53.274383 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8bf042-1ab0-40c2-acbc-ca46a062bcd0 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:53.281026 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2833d3b2-f45f-4dec-99af-055059bd13d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:53.281604 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:53.282518 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2833d3b2-f45f-4dec-99af-055059bd13d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:53.282809 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2833d3b2-f45f-4dec-99af-055059bd13d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:53.283165 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2833d3b2-f45f-4dec-99af-055059bd13d6 admin admin] PATH_INFO: `/v3/users/f72c79adfb294d11b42f5b0465e3014d` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:53.284238 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3be1f79f-beca-4070-ba94-2b3367a98b8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97f671de92d240ae85ca4893d1ff04e5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:53.284785 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2833d3b2-f45f-4dec-99af-055059bd13d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f72c79adfb294d11b42f5b0465e3014d)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:53.287421 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2833d3b2-f45f-4dec-99af-055059bd13d6 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:53.310189 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1339/2673] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:36:53 2022] DELETE /identity/v3/users/97f671de92d240ae85ca4893d1ff04e5 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:36:53.354950 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2833d3b2-f45f-4dec-99af-055059bd13d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f72c79adfb294d11b42f5b0465e3014d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:53.369661 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2833d3b2-f45f-4dec-99af-055059bd13d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f72c79adfb294d11b42f5b0465e3014d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:53.370646 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2833d3b2-f45f-4dec-99af-055059bd13d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f72c79adfb294d11b42f5b0465e3014d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:53.385788 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2833d3b2-f45f-4dec-99af-055059bd13d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f72c79adfb294d11b42f5b0465e3014d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:53.386591 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2833d3b2-f45f-4dec-99af-055059bd13d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f72c79adfb294d11b42f5b0465e3014d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:53.404189 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1335/2674] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:36:53 2022] DELETE /identity/v3/users/f72c79adfb294d11b42f5b0465e3014d => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:36:53.409071 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312ea08-e680-46b2-a0fb-8fcdd4022ad9 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:53.409401 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312ea08-e680-46b2-a0fb-8fcdd4022ad9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:53.409637 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312ea08-e680-46b2-a0fb-8fcdd4022ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:53.430410 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e312ea08-e680-46b2-a0fb-8fcdd4022ad9 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:53.431651 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e312ea08-e680-46b2-a0fb-8fcdd4022ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:53.493525 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1340/2675] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:53.500481 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2833d3b2-f45f-4dec-99af-055059bd13d6 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:53.501980 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2833d3b2-f45f-4dec-99af-055059bd13d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:53.507245 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e312ea08-e680-46b2-a0fb-8fcdd4022ad9 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:53.508588 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e312ea08-e680-46b2-a0fb-8fcdd4022ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:53.521491 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2833d3b2-f45f-4dec-99af-055059bd13d6 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:53.526567 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dada975d-0ff3-4770-ac69-24f4c73d144a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:53.527265 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dada975d-0ff3-4770-ac69-24f4c73d144a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:53.527609 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dada975d-0ff3-4770-ac69-24f4c73d144a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:53.528081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dada975d-0ff3-4770-ac69-24f4c73d144a admin admin] PATH_INFO: `/v3/projects/36a1fa6b4a754e47afb99e9b237ea400` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:53.540778 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dada975d-0ff3-4770-ac69-24f4c73d144a admin admin] RBAC: Authorizing `identity:delete_project(project_id=36a1fa6b4a754e47afb99e9b237ea400)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:53.542832 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dada975d-0ff3-4770-ac69-24f4c73d144a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:53.575508 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e312ea08-e680-46b2-a0fb-8fcdd4022ad9 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:53.579520 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc210585-24ab-4ad1-a300-66b93f35a956 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:53.580266 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc210585-24ab-4ad1-a300-66b93f35a956 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:53.580531 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc210585-24ab-4ad1-a300-66b93f35a956 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:53.580801 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc210585-24ab-4ad1-a300-66b93f35a956 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:53.590180 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc210585-24ab-4ad1-a300-66b93f35a956 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:53.591539 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dada975d-0ff3-4770-ac69-24f4c73d144a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36a1fa6b4a754e47afb99e9b237ea400', 'request_id': 'req-dada975d-0ff3-4770-ac69-24f4c73d144a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:53.591759 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc210585-24ab-4ad1-a300-66b93f35a956 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:53.597122 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dada975d-0ff3-4770-ac69-24f4c73d144a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36a1fa6b4a754e47afb99e9b237ea400', 'request_id': 'req-dada975d-0ff3-4770-ac69-24f4c73d144a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:53.601455 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1341/2676] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:36:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:36:53.610242 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dada975d-0ff3-4770-ac69-24f4c73d144a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36a1fa6b4a754e47afb99e9b237ea400', 'request_id': 'req-dada975d-0ff3-4770-ac69-24f4c73d144a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:53.622658 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dada975d-0ff3-4770-ac69-24f4c73d144a admin admin] The token cache is being invalidate because project 36a1fa6b4a754e47afb99e9b237ea400 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:36:53.623089 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dada975d-0ff3-4770-ac69-24f4c73d144a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:53.623779 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dada975d-0ff3-4770-ac69-24f4c73d144a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:53.625315 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1336/2677] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:36:53 2022] DELETE /identity/v3/projects/36a1fa6b4a754e47afb99e9b237ea400 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:53.630511 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f6524-2388-47df-bb38-ee507b8a5931 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:53.630795 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f6524-2388-47df-bb38-ee507b8a5931 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:53.631104 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f6524-2388-47df-bb38-ee507b8a5931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:53.652187 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-126f6524-2388-47df-bb38-ee507b8a5931 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:53.653392 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-126f6524-2388-47df-bb38-ee507b8a5931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:53.722508 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1342/2678] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:53.730784 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630be38-7fcf-4e86-af76-a58a0d049671 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:53.731901 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630be38-7fcf-4e86-af76-a58a0d049671 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:53.731901 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630be38-7fcf-4e86-af76-a58a0d049671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:53.752015 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-8630be38-7fcf-4e86-af76-a58a0d049671 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:53.753447 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8630be38-7fcf-4e86-af76-a58a0d049671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:53.762555 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1337/2679] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:53.766887 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126f6524-2388-47df-bb38-ee507b8a5931 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:53.776912 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126f6524-2388-47df-bb38-ee507b8a5931 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:53.781862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97427624-0f1c-40c2-b39e-60555dc359c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:53.782495 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97427624-0f1c-40c2-b39e-60555dc359c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:53.782776 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97427624-0f1c-40c2-b39e-60555dc359c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:53.783097 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97427624-0f1c-40c2-b39e-60555dc359c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:53.783930 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97427624-0f1c-40c2-b39e-60555dc359c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:36:53.784239 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97427624-0f1c-40c2-b39e-60555dc359c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:53.785886 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97427624-0f1c-40c2-b39e-60555dc359c5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:53.799294 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1343/2680] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:36:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:36:53.803825 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8630be38-7fcf-4e86-af76-a58a0d049671 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:53.813617 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8630be38-7fcf-4e86-af76-a58a0d049671 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:53.818277 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c782913-8206-4bdd-a059-d21c48988bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:53.818890 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c782913-8206-4bdd-a059-d21c48988bda admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:53.819194 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c782913-8206-4bdd-a059-d21c48988bda admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:53.819461 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c782913-8206-4bdd-a059-d21c48988bda admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:53.820161 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c782913-8206-4bdd-a059-d21c48988bda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:53.822033 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c782913-8206-4bdd-a059-d21c48988bda admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:53.855449 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1338/2681] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:36:53 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:36:53.859826 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97427624-0f1c-40c2-b39e-60555dc359c5 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:53.871617 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97427624-0f1c-40c2-b39e-60555dc359c5 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:53.877342 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:53.894053 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c782913-8206-4bdd-a059-d21c48988bda admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:53.898405 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1c782913-8206-4bdd-a059-d21c48988bda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:53.914377 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:53.942013 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f0171-40c5-4fb0-a903-5c3783620c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:53.942642 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f0171-40c5-4fb0-a903-5c3783620c50 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:53.942939 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f0171-40c5-4fb0-a903-5c3783620c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:53.943203 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f0171-40c5-4fb0-a903-5c3783620c50 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:53.943868 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4f0171-40c5-4fb0-a903-5c3783620c50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:53.945787 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4f0171-40c5-4fb0-a903-5c3783620c50 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:53.967850 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1344/2682] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:36:53 2022] POST /identity/v3/users => generated 290 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:36:53.971901 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f0171-40c5-4fb0-a903-5c3783620c50 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:53.983765 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f0171-40c5-4fb0-a903-5c3783620c50 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:53.986858 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c782913-8206-4bdd-a059-d21c48988bda admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:53.989381 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42c00ca-d011-4f66-afa8-a03e0c7d9031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:53.990112 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c00ca-d011-4f66-afa8-a03e0c7d9031 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:53.990502 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c00ca-d011-4f66-afa8-a03e0c7d9031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:53.990878 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c00ca-d011-4f66-afa8-a03e0c7d9031 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:53.991706 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42c00ca-d011-4f66-afa8-a03e0c7d9031 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:53.991895 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e46e6-72e4-4d8d-bc2d-601a33ea6443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:53.992486 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e46e6-72e4-4d8d-bc2d-601a33ea6443 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:53.992765 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e46e6-72e4-4d8d-bc2d-601a33ea6443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:53.993037 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e46e6-72e4-4d8d-bc2d-601a33ea6443 admin admin] PATH_INFO: `/v3/projects/72f8d9c76a5e4d1682113aaca1652d74` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:53.993800 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42c00ca-d011-4f66-afa8-a03e0c7d9031 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:53.994251 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58e46e6-72e4-4d8d-bc2d-601a33ea6443 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72f8d9c76a5e4d1682113aaca1652d74)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:53.996177 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58e46e6-72e4-4d8d-bc2d-601a33ea6443 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:54.006406 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1345/2683] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 10:36:54.010463 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42c00ca-d011-4f66-afa8-a03e0c7d9031 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:54.021125 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42c00ca-d011-4f66-afa8-a03e0c7d9031 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:54.026579 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663ac61f-f6c1-4ba7-b1c2-71db90df10fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:54.027240 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ac61f-f6c1-4ba7-b1c2-71db90df10fc admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:54.027590 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ac61f-f6c1-4ba7-b1c2-71db90df10fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:54.027873 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ac61f-f6c1-4ba7-b1c2-71db90df10fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:54.028567 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663ac61f-f6c1-4ba7-b1c2-71db90df10fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:54.030163 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663ac61f-f6c1-4ba7-b1c2-71db90df10fc admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:54.040981 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1346/2684] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:36:54.045430 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663ac61f-f6c1-4ba7-b1c2-71db90df10fc admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:54.048578 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d58e46e6-72e4-4d8d-bc2d-601a33ea6443 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72f8d9c76a5e4d1682113aaca1652d74', 'request_id': 'req-d58e46e6-72e4-4d8d-bc2d-601a33ea6443'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:54.054473 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d58e46e6-72e4-4d8d-bc2d-601a33ea6443 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72f8d9c76a5e4d1682113aaca1652d74', 'request_id': 'req-d58e46e6-72e4-4d8d-bc2d-601a33ea6443'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:54.055704 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663ac61f-f6c1-4ba7-b1c2-71db90df10fc admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:54.064461 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d58e46e6-72e4-4d8d-bc2d-601a33ea6443 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72f8d9c76a5e4d1682113aaca1652d74', 'request_id': 'req-d58e46e6-72e4-4d8d-bc2d-601a33ea6443'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:54.076625 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d58e46e6-72e4-4d8d-bc2d-601a33ea6443 admin admin] The token cache is being invalidate because project 72f8d9c76a5e4d1682113aaca1652d74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:36:54.076916 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d58e46e6-72e4-4d8d-bc2d-601a33ea6443 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:54.077627 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d58e46e6-72e4-4d8d-bc2d-601a33ea6443 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:54.079237 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1339/2685] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:36:53 2022] DELETE /identity/v3/projects/72f8d9c76a5e4d1682113aaca1652d74 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:54.083095 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e46e6-72e4-4d8d-bc2d-601a33ea6443 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:54.084552 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d58e46e6-72e4-4d8d-bc2d-601a33ea6443 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:54.118626 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e07b9d1-c277-4cfd-8a75-2d357214a832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:54.119354 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e07b9d1-c277-4cfd-8a75-2d357214a832 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:54.119638 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e07b9d1-c277-4cfd-8a75-2d357214a832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:54.119916 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e07b9d1-c277-4cfd-8a75-2d357214a832 admin admin] PATH_INFO: `/v3/projects/b23a59f0fc5741d4bb3fb7726e6ab437/users/181bc520411e4f64b66b4351c37689d5/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:54.134839 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e07b9d1-c277-4cfd-8a75-2d357214a832 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b23a59f0fc5741d4bb3fb7726e6ab437, user_id=181bc520411e4f64b66b4351c37689d5, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:54.137903 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e07b9d1-c277-4cfd-8a75-2d357214a832 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:54.148339 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e46e6-72e4-4d8d-bc2d-601a33ea6443 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:54.152624 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1347/2686] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:36:54 2022] PUT /identity/v3/projects/b23a59f0fc5741d4bb3fb7726e6ab437/users/181bc520411e4f64b66b4351c37689d5/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:36:54.164769 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8ef60-a1b4-4b9b-9c64-42dc26f0f0e3 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:54.165059 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8ef60-a1b4-4b9b-9c64-42dc26f0f0e3 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:54.165336 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8ef60-a1b4-4b9b-9c64-42dc26f0f0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:54.193827 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-45c8ef60-a1b4-4b9b-9c64-42dc26f0f0e3 None None] MFA Rules not processed for user `181bc520411e4f64b66b4351c37689d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:54.195089 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-45c8ef60-a1b4-4b9b-9c64-42dc26f0f0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:54.195521 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:54.213679 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d69e9-a4dc-49da-986f-7f245a00bdcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:54.214331 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d69e9-a4dc-49da-986f-7f245a00bdcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:54.214612 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d69e9-a4dc-49da-986f-7f245a00bdcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:54.214931 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d69e9-a4dc-49da-986f-7f245a00bdcb admin admin] PATH_INFO: `/v3/users/3040e9c551434b53b548e4645b948ec5` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:54.216219 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279d69e9-a4dc-49da-986f-7f245a00bdcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3040e9c551434b53b548e4645b948ec5)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:54.218178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279d69e9-a4dc-49da-986f-7f245a00bdcb admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:54.249486 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:54.270669 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1348/2687] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:36:54.271711 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-279d69e9-a4dc-49da-986f-7f245a00bdcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3040e9c551434b53b548e4645b948ec5'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:54.275426 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8ef60-a1b4-4b9b-9c64-42dc26f0f0e3 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:54.276725 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-45c8ef60-a1b4-4b9b-9c64-42dc26f0f0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:54.280432 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-279d69e9-a4dc-49da-986f-7f245a00bdcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3040e9c551434b53b548e4645b948ec5'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:54.281025 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-279d69e9-a4dc-49da-986f-7f245a00bdcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3040e9c551434b53b548e4645b948ec5'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:54.290392 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8ef60-a1b4-4b9b-9c64-42dc26f0f0e3 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:54.293766 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-279d69e9-a4dc-49da-986f-7f245a00bdcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3040e9c551434b53b548e4645b948ec5'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:54.294552 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-279d69e9-a4dc-49da-986f-7f245a00bdcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3040e9c551434b53b548e4645b948ec5'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:54.295327 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a89356-9d6b-45b7-9f34-d504c49b8657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:54.295947 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a89356-9d6b-45b7-9f34-d504c49b8657 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:54.296229 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a89356-9d6b-45b7-9f34-d504c49b8657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:54.296786 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a89356-9d6b-45b7-9f34-d504c49b8657 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:54.297160 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a89356-9d6b-45b7-9f34-d504c49b8657 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:54.299084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a89356-9d6b-45b7-9f34-d504c49b8657 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:54.307821 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1340/2688] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:36:54 2022] DELETE /identity/v3/users/3040e9c551434b53b548e4645b948ec5 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:36:54.329374 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1349/2689] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:36:54 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:36:54.333458 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d69e9-a4dc-49da-986f-7f245a00bdcb admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:54.334914 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-279d69e9-a4dc-49da-986f-7f245a00bdcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:54.404585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d69e9-a4dc-49da-986f-7f245a00bdcb admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:54.470224 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a2400-8ee0-4e66-ad59-73001f90cc2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:54.470890 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a2400-8ee0-4e66-ad59-73001f90cc2a admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:54.471200 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a2400-8ee0-4e66-ad59-73001f90cc2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:54.471476 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a2400-8ee0-4e66-ad59-73001f90cc2a admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:54.472156 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817a2400-8ee0-4e66-ad59-73001f90cc2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:54.474753 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817a2400-8ee0-4e66-ad59-73001f90cc2a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:54.492058 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:54.498337 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1341/2690] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:36:54 2022] POST /identity/v3/users => generated 297 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:36:54.502537 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a89356-9d6b-45b7-9f34-d504c49b8657 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:54.513238 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a89356-9d6b-45b7-9f34-d504c49b8657 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:54.518248 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f95c3-24c7-4abf-b762-74565d321a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:54.518874 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f95c3-24c7-4abf-b762-74565d321a62 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:54.519186 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f95c3-24c7-4abf-b762-74565d321a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:54.519464 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f95c3-24c7-4abf-b762-74565d321a62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:54.520161 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47f95c3-24c7-4abf-b762-74565d321a62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:54.521771 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47f95c3-24c7-4abf-b762-74565d321a62 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:54.535637 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1350/2691] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:36:54.536914 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a2400-8ee0-4e66-ad59-73001f90cc2a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:54.547465 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a2400-8ee0-4e66-ad59-73001f90cc2a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:54.552284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662ac2d-8cf0-47b2-ac3b-5867f568fb72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:54.552962 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662ac2d-8cf0-47b2-ac3b-5867f568fb72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:54.553220 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662ac2d-8cf0-47b2-ac3b-5867f568fb72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:54.553510 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662ac2d-8cf0-47b2-ac3b-5867f568fb72 admin admin] PATH_INFO: `/v3/projects/b801461e77bb445d8dd4e17c63876be6/users/490f275f31ee46d4909b38973f625f00/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:54.569153 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9662ac2d-8cf0-47b2-ac3b-5867f568fb72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b801461e77bb445d8dd4e17c63876be6, user_id=490f275f31ee46d4909b38973f625f00, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:54.572208 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9662ac2d-8cf0-47b2-ac3b-5867f568fb72 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:54.587427 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1342/2692] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:36:54 2022] PUT /identity/v3/projects/b801461e77bb445d8dd4e17c63876be6/users/490f275f31ee46d4909b38973f625f00/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:54.588427 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f95c3-24c7-4abf-b762-74565d321a62 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:54.590451 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e47f95c3-24c7-4abf-b762-74565d321a62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:54.591044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662ac2d-8cf0-47b2-ac3b-5867f568fb72 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:54.608166 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f95c3-24c7-4abf-b762-74565d321a62 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:54.655145 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662ac2d-8cf0-47b2-ac3b-5867f568fb72 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:54.660240 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b2d8c-472b-4330-ad69-f9443a8be4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:54.660866 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b2d8c-472b-4330-ad69-f9443a8be4ed admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:54.661164 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b2d8c-472b-4330-ad69-f9443a8be4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:54.661424 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b2d8c-472b-4330-ad69-f9443a8be4ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:54.662179 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520b2d8c-472b-4330-ad69-f9443a8be4ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:54.663821 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520b2d8c-472b-4330-ad69-f9443a8be4ed admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:54.674430 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1343/2693] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:36:54.675588 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c670d-3046-4050-a97e-e7bf4d702d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:54.676295 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c670d-3046-4050-a97e-e7bf4d702d6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:54.676580 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c670d-3046-4050-a97e-e7bf4d702d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:54.676853 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c670d-3046-4050-a97e-e7bf4d702d6b admin admin] PATH_INFO: `/v3/projects/24987d43a2784320b9746268c2193144` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:54.678175 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c670d-3046-4050-a97e-e7bf4d702d6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=24987d43a2784320b9746268c2193144)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:54.679161 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b2d8c-472b-4330-ad69-f9443a8be4ed admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:54.680131 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c670d-3046-4050-a97e-e7bf4d702d6b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:54.689746 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b2d8c-472b-4330-ad69-f9443a8be4ed admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:54.694759 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db3878-b692-4f1f-a104-e3a26dcee8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:54.695455 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db3878-b692-4f1f-a104-e3a26dcee8c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:54.695738 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db3878-b692-4f1f-a104-e3a26dcee8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:54.696011 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db3878-b692-4f1f-a104-e3a26dcee8c7 admin admin] PATH_INFO: `/v3/projects/b801461e77bb445d8dd4e17c63876be6/users/490f275f31ee46d4909b38973f625f00/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:54.698065 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db3878-b692-4f1f-a104-e3a26dcee8c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b801461e77bb445d8dd4e17c63876be6, user_id=490f275f31ee46d4909b38973f625f00, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:54.700726 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db3878-b692-4f1f-a104-e3a26dcee8c7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:54.711349 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1344/2694] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:36:54 2022] PUT /identity/v3/projects/b801461e77bb445d8dd4e17c63876be6/users/490f275f31ee46d4909b38973f625f00/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:54.721748 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d5a28-8ca7-42fa-a9e9-f52d7ad2f329 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:54.722043 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d5a28-8ca7-42fa-a9e9-f52d7ad2f329 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:54.722451 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d5a28-8ca7-42fa-a9e9-f52d7ad2f329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:54.730354 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-de8c670d-3046-4050-a97e-e7bf4d702d6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24987d43a2784320b9746268c2193144', 'request_id': 'req-de8c670d-3046-4050-a97e-e7bf4d702d6b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:54.739709 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-de8c670d-3046-4050-a97e-e7bf4d702d6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24987d43a2784320b9746268c2193144', 'request_id': 'req-de8c670d-3046-4050-a97e-e7bf4d702d6b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:54.746464 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-de8c670d-3046-4050-a97e-e7bf4d702d6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24987d43a2784320b9746268c2193144', 'request_id': 'req-de8c670d-3046-4050-a97e-e7bf4d702d6b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:54.750386 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a71d5a28-8ca7-42fa-a9e9-f52d7ad2f329 None None] MFA Rules not processed for user `490f275f31ee46d4909b38973f625f00`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:54.751652 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a71d5a28-8ca7-42fa-a9e9-f52d7ad2f329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:54.758155 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-de8c670d-3046-4050-a97e-e7bf4d702d6b admin admin] The token cache is being invalidate because project 24987d43a2784320b9746268c2193144 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:36:54.758440 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-de8c670d-3046-4050-a97e-e7bf4d702d6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:54.759138 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-de8c670d-3046-4050-a97e-e7bf4d702d6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:36:54.760701 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1351/2695] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:36:54 2022] DELETE /identity/v3/projects/24987d43a2784320b9746268c2193144 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:36:54.766162 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c16a3-ed9b-4a75-a695-c50e29034a23 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:54.766442 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c16a3-ed9b-4a75-a695-c50e29034a23 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:54.766713 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c16a3-ed9b-4a75-a695-c50e29034a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:54.787243 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-845c16a3-ed9b-4a75-a695-c50e29034a23 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:54.788460 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-845c16a3-ed9b-4a75-a695-c50e29034a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:54.824775 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1345/2696] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:54 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:36:54.831533 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:54.837197 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71d5a28-8ca7-42fa-a9e9-f52d7ad2f329 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:54.838856 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a71d5a28-8ca7-42fa-a9e9-f52d7ad2f329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:54.851827 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1352/2697] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:36:54.859730 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a4304-103b-43bf-9185-49b3fffd10fd None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:54.860024 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a4304-103b-43bf-9185-49b3fffd10fd None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:54.860288 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a4304-103b-43bf-9185-49b3fffd10fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:54.880261 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-4f1a4304-103b-43bf-9185-49b3fffd10fd None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:54.881566 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4f1a4304-103b-43bf-9185-49b3fffd10fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:54.886023 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:54.890605 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1353/2698] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:54.895014 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1a4304-103b-43bf-9185-49b3fffd10fd None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:54.901089 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71d5a28-8ca7-42fa-a9e9-f52d7ad2f329 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:54.904308 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1a4304-103b-43bf-9185-49b3fffd10fd None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:54.904791 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca1801a-71e7-45b4-9ce7-eb2b9c396a28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:54.905422 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca1801a-71e7-45b4-9ce7-eb2b9c396a28 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:54.905702 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca1801a-71e7-45b4-9ce7-eb2b9c396a28 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:54.905982 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca1801a-71e7-45b4-9ce7-eb2b9c396a28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:54.909143 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd6a977-602d-4a2f-95f0-288021c73dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:54.909745 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd6a977-602d-4a2f-95f0-288021c73dad admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:54.910020 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd6a977-602d-4a2f-95f0-288021c73dad admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:54.910286 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd6a977-602d-4a2f-95f0-288021c73dad admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:54.910993 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd6a977-602d-4a2f-95f0-288021c73dad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:36:54.911295 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd6a977-602d-4a2f-95f0-288021c73dad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:54.912880 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd6a977-602d-4a2f-95f0-288021c73dad admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:54.915445 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca1801a-71e7-45b4-9ce7-eb2b9c396a28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:54.916921 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca1801a-71e7-45b4-9ce7-eb2b9c396a28 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:54.928658 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1354/2699] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:36:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:36:54.928872 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1346/2700] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:36:54 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:36:54.929825 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd6a977-602d-4a2f-95f0-288021c73dad admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:54.940046 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd6a977-602d-4a2f-95f0-288021c73dad admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:54.945315 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bb9f9a-1fb6-4053-8edb-5d2e04deebc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:54.945937 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bb9f9a-1fb6-4053-8edb-5d2e04deebc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:54.946267 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bb9f9a-1fb6-4053-8edb-5d2e04deebc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:54.946509 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bb9f9a-1fb6-4053-8edb-5d2e04deebc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:54.947250 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bb9f9a-1fb6-4053-8edb-5d2e04deebc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:54.949300 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bb9f9a-1fb6-4053-8edb-5d2e04deebc7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:54.981730 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1355/2701] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:36:54 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:36:54.986308 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca1801a-71e7-45b4-9ce7-eb2b9c396a28 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:54.997750 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca1801a-71e7-45b4-9ce7-eb2b9c396a28 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:55.009511 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bb9f9a-1fb6-4053-8edb-5d2e04deebc7 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:55.057638 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e177d0-6839-44cf-8326-7769dc8dc14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:55.058268 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e177d0-6839-44cf-8326-7769dc8dc14a admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:55.058551 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e177d0-6839-44cf-8326-7769dc8dc14a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:55.058867 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e177d0-6839-44cf-8326-7769dc8dc14a admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:55.059553 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e177d0-6839-44cf-8326-7769dc8dc14a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:55.061488 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e177d0-6839-44cf-8326-7769dc8dc14a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:55.069889 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bb9f9a-1fb6-4053-8edb-5d2e04deebc7 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:55.073943 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf2688a-2805-49a9-a388-f8b5646f377f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:55.074883 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf2688a-2805-49a9-a388-f8b5646f377f service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:55.075300 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf2688a-2805-49a9-a388-f8b5646f377f service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:55.075608 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf2688a-2805-49a9-a388-f8b5646f377f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:55.077564 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bcf2688a-2805-49a9-a388-f8b5646f377f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:55.084567 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1347/2702] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:36:54 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:36:55.088823 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e177d0-6839-44cf-8326-7769dc8dc14a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:55.098666 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e177d0-6839-44cf-8326-7769dc8dc14a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:55.103736 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b990a-4705-477f-a130-b3156213c980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:55.104335 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b990a-4705-477f-a130-b3156213c980 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:55.104619 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b990a-4705-477f-a130-b3156213c980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:55.104905 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b990a-4705-477f-a130-b3156213c980 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:55.105655 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841b990a-4705-477f-a130-b3156213c980 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:55.107292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841b990a-4705-477f-a130-b3156213c980 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:55.119129 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1348/2703] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 10:36:55.121449 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b990a-4705-477f-a130-b3156213c980 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:55.131629 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b990a-4705-477f-a130-b3156213c980 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:55.136445 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c32479-f096-44dd-908c-6367523a77be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:55.136928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf2688a-2805-49a9-a388-f8b5646f377f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:55.137498 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c32479-f096-44dd-908c-6367523a77be admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:55.137903 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c32479-f096-44dd-908c-6367523a77be admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:55.138336 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c32479-f096-44dd-908c-6367523a77be admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:55.138447 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf2688a-2805-49a9-a388-f8b5646f377f service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:55.139557 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c32479-f096-44dd-908c-6367523a77be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:55.141091 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c32479-f096-44dd-908c-6367523a77be admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:55.155662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1349/2704] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:36:55.155911 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1356/2705] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:36:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 10:36:55.160157 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c32479-f096-44dd-908c-6367523a77be admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:55.173870 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c32479-f096-44dd-908c-6367523a77be admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:55.179600 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57df355-2e9e-4345-b7bd-4a206996f461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:55.180466 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57df355-2e9e-4345-b7bd-4a206996f461 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:55.180752 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57df355-2e9e-4345-b7bd-4a206996f461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:55.181026 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57df355-2e9e-4345-b7bd-4a206996f461 admin admin] PATH_INFO: `/v3/projects/dde251794e4b4ea4ae125a8f876cbe3b/users/1beabcf3cdc143ecb95b765a46c0c60f/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:55.196745 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57df355-2e9e-4345-b7bd-4a206996f461 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dde251794e4b4ea4ae125a8f876cbe3b, user_id=1beabcf3cdc143ecb95b765a46c0c60f, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:55.199610 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57df355-2e9e-4345-b7bd-4a206996f461 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:55.215654 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1350/2706] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:36:55 2022] PUT /identity/v3/projects/dde251794e4b4ea4ae125a8f876cbe3b/users/1beabcf3cdc143ecb95b765a46c0c60f/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:36:55.227254 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ce6ee5-bde8-41b8-beed-6ec38cc73682 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:55.227588 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ce6ee5-bde8-41b8-beed-6ec38cc73682 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:55.227869 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ce6ee5-bde8-41b8-beed-6ec38cc73682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:55.257188 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-58ce6ee5-bde8-41b8-beed-6ec38cc73682 None None] MFA Rules not processed for user `1beabcf3cdc143ecb95b765a46c0c60f`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:36:55.258535 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-58ce6ee5-bde8-41b8-beed-6ec38cc73682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:36:55.313392 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:36:55.332531 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1357/2707] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:36:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:36:55.343930 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57df355-2e9e-4345-b7bd-4a206996f461 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:36:55.400848 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57df355-2e9e-4345-b7bd-4a206996f461 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:36:55.405030 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596ced0-54aa-4c3e-b0bb-23b5ab47d5c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:36:55.405643 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596ced0-54aa-4c3e-b0bb-23b5ab47d5c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:36:55.405937 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596ced0-54aa-4c3e-b0bb-23b5ab47d5c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:36:55.406188 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596ced0-54aa-4c3e-b0bb-23b5ab47d5c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:36:55.414860 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c596ced0-54aa-4c3e-b0bb-23b5ab47d5c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:36:55.416296 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c596ced0-54aa-4c3e-b0bb-23b5ab47d5c8 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:36:55.425244 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1351/2708] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:36:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:37:05.187260 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ce6ee5-bde8-41b8-beed-6ec38cc73682 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:05.188882 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-58ce6ee5-bde8-41b8-beed-6ec38cc73682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:37:05.254534 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ce6ee5-bde8-41b8-beed-6ec38cc73682 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:05.259735 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c701a9f-e5e1-416c-82ce-198fa517211a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:05.260386 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c701a9f-e5e1-416c-82ce-198fa517211a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:05.260683 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c701a9f-e5e1-416c-82ce-198fa517211a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:05.260938 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c701a9f-e5e1-416c-82ce-198fa517211a admin admin] PATH_INFO: `/v3/users/181bc520411e4f64b66b4351c37689d5` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:05.262091 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c701a9f-e5e1-416c-82ce-198fa517211a admin admin] RBAC: Authorizing `identity:delete_user(user_id=181bc520411e4f64b66b4351c37689d5)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:05.264396 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c701a9f-e5e1-416c-82ce-198fa517211a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:05.319236 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1c701a9f-e5e1-416c-82ce-198fa517211a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '181bc520411e4f64b66b4351c37689d5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:05.319946 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1c701a9f-e5e1-416c-82ce-198fa517211a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '181bc520411e4f64b66b4351c37689d5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:05.320578 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1c701a9f-e5e1-416c-82ce-198fa517211a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '181bc520411e4f64b66b4351c37689d5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:05.329937 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1c701a9f-e5e1-416c-82ce-198fa517211a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '181bc520411e4f64b66b4351c37689d5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:05.337045 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1c701a9f-e5e1-416c-82ce-198fa517211a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '181bc520411e4f64b66b4351c37689d5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:05.356213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1358/2709] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:37:05 2022] DELETE /identity/v3/users/181bc520411e4f64b66b4351c37689d5 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:37:05.360960 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772b3ff-9b5d-4841-a4c5-7ab9e88ca602 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:05.361460 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772b3ff-9b5d-4841-a4c5-7ab9e88ca602 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:05.361558 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772b3ff-9b5d-4841-a4c5-7ab9e88ca602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:05.381585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-3772b3ff-9b5d-4841-a4c5-7ab9e88ca602 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:37:05.382895 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3772b3ff-9b5d-4841-a4c5-7ab9e88ca602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:37:05.412189 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:37:05.448705 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1352/2710] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:37:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:37:05.459304 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c701a9f-e5e1-416c-82ce-198fa517211a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:05.460826 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1c701a9f-e5e1-416c-82ce-198fa517211a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:37:05.519816 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c701a9f-e5e1-416c-82ce-198fa517211a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:05.523789 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6b204b-a24f-4935-90a0-1bc4ce258d97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:05.524410 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b204b-a24f-4935-90a0-1bc4ce258d97 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:05.524689 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b204b-a24f-4935-90a0-1bc4ce258d97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:05.524963 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b204b-a24f-4935-90a0-1bc4ce258d97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:05.534361 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6b204b-a24f-4935-90a0-1bc4ce258d97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:05.535825 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6b204b-a24f-4935-90a0-1bc4ce258d97 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:05.545432 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1359/2711] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:37:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:37:05.832637 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3772b3ff-9b5d-4841-a4c5-7ab9e88ca602 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:05.834096 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3772b3ff-9b5d-4841-a4c5-7ab9e88ca602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:37:05.846655 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3772b3ff-9b5d-4841-a4c5-7ab9e88ca602 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:05.852164 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b86530-e5b0-4799-b1f6-f3346a0c1f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:05.853051 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b86530-e5b0-4799-b1f6-f3346a0c1f66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:05.853336 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b86530-e5b0-4799-b1f6-f3346a0c1f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:05.853711 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b86530-e5b0-4799-b1f6-f3346a0c1f66 admin admin] PATH_INFO: `/v3/projects/b23a59f0fc5741d4bb3fb7726e6ab437` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:05.855039 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b86530-e5b0-4799-b1f6-f3346a0c1f66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b23a59f0fc5741d4bb3fb7726e6ab437)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:05.857100 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b86530-e5b0-4799-b1f6-f3346a0c1f66 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:05.907305 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-71b86530-e5b0-4799-b1f6-f3346a0c1f66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b23a59f0fc5741d4bb3fb7726e6ab437', 'request_id': 'req-71b86530-e5b0-4799-b1f6-f3346a0c1f66'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:05.913580 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-71b86530-e5b0-4799-b1f6-f3346a0c1f66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b23a59f0fc5741d4bb3fb7726e6ab437', 'request_id': 'req-71b86530-e5b0-4799-b1f6-f3346a0c1f66'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:05.923874 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-71b86530-e5b0-4799-b1f6-f3346a0c1f66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b23a59f0fc5741d4bb3fb7726e6ab437', 'request_id': 'req-71b86530-e5b0-4799-b1f6-f3346a0c1f66'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:05.939603 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-71b86530-e5b0-4799-b1f6-f3346a0c1f66 admin admin] The token cache is being invalidate because project b23a59f0fc5741d4bb3fb7726e6ab437 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:37:05.939894 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-71b86530-e5b0-4799-b1f6-f3346a0c1f66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:05.940623 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-71b86530-e5b0-4799-b1f6-f3346a0c1f66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:05.942188 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1353/2712] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:37:05 2022] DELETE /identity/v3/projects/b23a59f0fc5741d4bb3fb7726e6ab437 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:37:05.946496 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6b204b-a24f-4935-90a0-1bc4ce258d97 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:05.957336 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6b204b-a24f-4935-90a0-1bc4ce258d97 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:05.966244 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:37:06.153798 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e762d8-a61a-4302-92ce-76719971ef43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:06.154300 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e762d8-a61a-4302-92ce-76719971ef43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:06.154598 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e762d8-a61a-4302-92ce-76719971ef43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:06.154856 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e762d8-a61a-4302-92ce-76719971ef43 admin admin] PATH_INFO: `/v3/users/490f275f31ee46d4909b38973f625f00` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:06.156248 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e762d8-a61a-4302-92ce-76719971ef43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=490f275f31ee46d4909b38973f625f00)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:06.158399 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e762d8-a61a-4302-92ce-76719971ef43 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:06.214153 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c9e762d8-a61a-4302-92ce-76719971ef43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '490f275f31ee46d4909b38973f625f00'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:06.214861 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c9e762d8-a61a-4302-92ce-76719971ef43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '490f275f31ee46d4909b38973f625f00'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:06.215606 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c9e762d8-a61a-4302-92ce-76719971ef43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '490f275f31ee46d4909b38973f625f00'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:06.225713 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c9e762d8-a61a-4302-92ce-76719971ef43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '490f275f31ee46d4909b38973f625f00'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:06.232825 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c9e762d8-a61a-4302-92ce-76719971ef43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '490f275f31ee46d4909b38973f625f00'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:06.254489 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1360/2713] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:37:05 2022] DELETE /identity/v3/users/490f275f31ee46d4909b38973f625f00 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:37:06.539666 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b86530-e5b0-4799-b1f6-f3346a0c1f66 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:06.541170 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-71b86530-e5b0-4799-b1f6-f3346a0c1f66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:37:06.560264 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b86530-e5b0-4799-b1f6-f3346a0c1f66 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:06.621547 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741182a6-437d-4f60-88aa-184d8d46a398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:06.622226 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741182a6-437d-4f60-88aa-184d8d46a398 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:06.622504 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741182a6-437d-4f60-88aa-184d8d46a398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:06.622772 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741182a6-437d-4f60-88aa-184d8d46a398 admin admin] PATH_INFO: `/v3/projects/b801461e77bb445d8dd4e17c63876be6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:06.623969 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741182a6-437d-4f60-88aa-184d8d46a398 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b801461e77bb445d8dd4e17c63876be6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:06.626110 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741182a6-437d-4f60-88aa-184d8d46a398 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:06.674448 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-741182a6-437d-4f60-88aa-184d8d46a398 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b801461e77bb445d8dd4e17c63876be6', 'request_id': 'req-741182a6-437d-4f60-88aa-184d8d46a398'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:06.679310 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:37:06.680879 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-741182a6-437d-4f60-88aa-184d8d46a398 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b801461e77bb445d8dd4e17c63876be6', 'request_id': 'req-741182a6-437d-4f60-88aa-184d8d46a398'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:06.690661 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-741182a6-437d-4f60-88aa-184d8d46a398 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b801461e77bb445d8dd4e17c63876be6', 'request_id': 'req-741182a6-437d-4f60-88aa-184d8d46a398'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:06.702628 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-741182a6-437d-4f60-88aa-184d8d46a398 admin admin] The token cache is being invalidate because project b801461e77bb445d8dd4e17c63876be6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:37:06.703003 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-741182a6-437d-4f60-88aa-184d8d46a398 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:06.703615 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-741182a6-437d-4f60-88aa-184d8d46a398 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:06.705226 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1354/2714] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:37:06 2022] DELETE /identity/v3/projects/b801461e77bb445d8dd4e17c63876be6 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:37:06.710762 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a8e33c-0609-4ff8-bec7-10ec892b458a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:06.711089 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a8e33c-0609-4ff8-bec7-10ec892b458a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:06.711406 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a8e33c-0609-4ff8-bec7-10ec892b458a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:06.736118 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e7a8e33c-0609-4ff8-bec7-10ec892b458a None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:37:06.737480 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e7a8e33c-0609-4ff8-bec7-10ec892b458a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:37:06.808581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1361/2715] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:37:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:37:06.817050 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b4e4b3-e027-4126-a9a1-3615ca3c022f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:06.817355 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b4e4b3-e027-4126-a9a1-3615ca3c022f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:06.817624 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b4e4b3-e027-4126-a9a1-3615ca3c022f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:06.843409 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-68b4e4b3-e027-4126-a9a1-3615ca3c022f None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:37:06.844894 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-68b4e4b3-e027-4126-a9a1-3615ca3c022f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:37:06.853168 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1355/2716] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:37:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:37:06.857801 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a8e33c-0609-4ff8-bec7-10ec892b458a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:06.868179 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a8e33c-0609-4ff8-bec7-10ec892b458a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:06.873545 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d70301-a811-48f0-a35e-5df4ca30d8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:06.874201 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d70301-a811-48f0-a35e-5df4ca30d8f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:06.874477 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d70301-a811-48f0-a35e-5df4ca30d8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:06.874745 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d70301-a811-48f0-a35e-5df4ca30d8f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:06.875678 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d70301-a811-48f0-a35e-5df4ca30d8f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:37:06.875963 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d70301-a811-48f0-a35e-5df4ca30d8f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:06.877568 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d70301-a811-48f0-a35e-5df4ca30d8f6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:06.889708 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1362/2717] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:37:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:37:06.894469 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b4e4b3-e027-4126-a9a1-3615ca3c022f None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:06.904149 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b4e4b3-e027-4126-a9a1-3615ca3c022f None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:06.910004 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330e9300-43ee-47d6-8e56-4982ceafd2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:06.910622 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330e9300-43ee-47d6-8e56-4982ceafd2e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:06.910931 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330e9300-43ee-47d6-8e56-4982ceafd2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:06.911225 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330e9300-43ee-47d6-8e56-4982ceafd2e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:06.911884 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330e9300-43ee-47d6-8e56-4982ceafd2e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:06.913806 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330e9300-43ee-47d6-8e56-4982ceafd2e3 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:06.952076 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1356/2718] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:37:06 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:37:06.956949 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d70301-a811-48f0-a35e-5df4ca30d8f6 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:06.967362 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d70301-a811-48f0-a35e-5df4ca30d8f6 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:06.979735 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:37:07.038094 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f910f78-8403-42e8-9fe4-be74a12d2b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:07.038713 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f910f78-8403-42e8-9fe4-be74a12d2b88 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:07.039034 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f910f78-8403-42e8-9fe4-be74a12d2b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:07.039316 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f910f78-8403-42e8-9fe4-be74a12d2b88 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:07.039980 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f910f78-8403-42e8-9fe4-be74a12d2b88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:07.041908 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f910f78-8403-42e8-9fe4-be74a12d2b88 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:07.068087 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1363/2719] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:37:06 2022] POST /identity/v3/users => generated 287 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:37:07.072459 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330e9300-43ee-47d6-8e56-4982ceafd2e3 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:07.083968 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330e9300-43ee-47d6-8e56-4982ceafd2e3 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:07.089006 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6542f63-62b9-4338-bb70-a1adf752c153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:07.089612 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6542f63-62b9-4338-bb70-a1adf752c153 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:07.089892 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6542f63-62b9-4338-bb70-a1adf752c153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:07.090151 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6542f63-62b9-4338-bb70-a1adf752c153 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:07.090845 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6542f63-62b9-4338-bb70-a1adf752c153 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:07.092474 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6542f63-62b9-4338-bb70-a1adf752c153 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:07.103067 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1357/2720] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:37:07.107240 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f910f78-8403-42e8-9fe4-be74a12d2b88 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:07.117703 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f910f78-8403-42e8-9fe4-be74a12d2b88 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:07.122932 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb89d717-8ebb-4327-8712-24fb34c85754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:07.123581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89d717-8ebb-4327-8712-24fb34c85754 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:07.123891 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89d717-8ebb-4327-8712-24fb34c85754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:07.124170 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89d717-8ebb-4327-8712-24fb34c85754 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:07.124840 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb89d717-8ebb-4327-8712-24fb34c85754 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:07.126570 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb89d717-8ebb-4327-8712-24fb34c85754 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:07.137367 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1364/2721] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:37:07.144769 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6542f63-62b9-4338-bb70-a1adf752c153 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:07.157593 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6542f63-62b9-4338-bb70-a1adf752c153 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:07.167550 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a0a2b1-8764-40f2-a94c-9414856d06da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:07.168304 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0a2b1-8764-40f2-a94c-9414856d06da admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:07.168589 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0a2b1-8764-40f2-a94c-9414856d06da admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:07.168868 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0a2b1-8764-40f2-a94c-9414856d06da admin admin] PATH_INFO: `/v3/projects/b935a59abdc04b90a26aa301ebfdfe5b/users/058557c7cd6f41758868f2df2c1fd165/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:07.184589 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a0a2b1-8764-40f2-a94c-9414856d06da admin admin] RBAC: Authorizing `identity:create_grant(project_id=b935a59abdc04b90a26aa301ebfdfe5b, user_id=058557c7cd6f41758868f2df2c1fd165, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:07.187266 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a0a2b1-8764-40f2-a94c-9414856d06da admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:07.203496 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1358/2722] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:37:07 2022] PUT /identity/v3/projects/b935a59abdc04b90a26aa301ebfdfe5b/users/058557c7cd6f41758868f2df2c1fd165/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:37:07.215611 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df40ae-9467-4a6d-b7c6-1d6f55e665b7 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:07.216139 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df40ae-9467-4a6d-b7c6-1d6f55e665b7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:07.216139 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df40ae-9467-4a6d-b7c6-1d6f55e665b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:07.244902 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-87df40ae-9467-4a6d-b7c6-1d6f55e665b7 None None] MFA Rules not processed for user `058557c7cd6f41758868f2df2c1fd165`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:37:07.246101 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-87df40ae-9467-4a6d-b7c6-1d6f55e665b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:37:07.315402 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1365/2723] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:37:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:37:07.320410 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a0a2b1-8764-40f2-a94c-9414856d06da admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:07.387562 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a0a2b1-8764-40f2-a94c-9414856d06da admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:07.393173 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439787ad-e92b-42b5-befa-da45f0c42049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:07.393787 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439787ad-e92b-42b5-befa-da45f0c42049 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:07.394062 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439787ad-e92b-42b5-befa-da45f0c42049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:07.394355 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439787ad-e92b-42b5-befa-da45f0c42049 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:07.395048 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439787ad-e92b-42b5-befa-da45f0c42049 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:07.396939 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439787ad-e92b-42b5-befa-da45f0c42049 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:07.427571 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1359/2724] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:37:07 2022] POST /identity/v3/projects => generated 373 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:37:07.433890 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87df40ae-9467-4a6d-b7c6-1d6f55e665b7 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:07.446588 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87df40ae-9467-4a6d-b7c6-1d6f55e665b7 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:07.511208 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039a8db8-ea0f-4e38-b67c-003d81cec753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:07.512199 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a8db8-ea0f-4e38-b67c-003d81cec753 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:07.513228 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a8db8-ea0f-4e38-b67c-003d81cec753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:07.513622 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a8db8-ea0f-4e38-b67c-003d81cec753 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:07.514564 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039a8db8-ea0f-4e38-b67c-003d81cec753 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:07.517358 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039a8db8-ea0f-4e38-b67c-003d81cec753 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:07.544919 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1366/2725] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:37:07 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:37:07.549376 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439787ad-e92b-42b5-befa-da45f0c42049 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:07.560529 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439787ad-e92b-42b5-befa-da45f0c42049 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:07.566717 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f2f07c-45a1-4e25-b145-c91452ddbf0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:07.567382 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f2f07c-45a1-4e25-b145-c91452ddbf0f admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:07.567662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f2f07c-45a1-4e25-b145-c91452ddbf0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:07.567933 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f2f07c-45a1-4e25-b145-c91452ddbf0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:07.568665 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f2f07c-45a1-4e25-b145-c91452ddbf0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:07.570281 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f2f07c-45a1-4e25-b145-c91452ddbf0f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:07.581284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1360/2726] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:37:07.585576 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039a8db8-ea0f-4e38-b67c-003d81cec753 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:07.596103 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039a8db8-ea0f-4e38-b67c-003d81cec753 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:07.601072 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004272f4-eb05-4413-97de-5ecb86a6edd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:07.601734 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004272f4-eb05-4413-97de-5ecb86a6edd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:07.602009 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004272f4-eb05-4413-97de-5ecb86a6edd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:07.602282 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004272f4-eb05-4413-97de-5ecb86a6edd5 admin admin] PATH_INFO: `/v3/projects/5dc6d1ba779d4da78cace9bf5d3e8821/users/88d6a4c9467b4da9920bf5693a62e425/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:07.615658 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:37:07.619233 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004272f4-eb05-4413-97de-5ecb86a6edd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dc6d1ba779d4da78cace9bf5d3e8821, user_id=88d6a4c9467b4da9920bf5693a62e425, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:07.622321 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004272f4-eb05-4413-97de-5ecb86a6edd5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:07.639011 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1367/2727] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:37:07 2022] PUT /identity/v3/projects/5dc6d1ba779d4da78cace9bf5d3e8821/users/88d6a4c9467b4da9920bf5693a62e425/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:37:07.643361 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f2f07c-45a1-4e25-b145-c91452ddbf0f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:07.654002 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f2f07c-45a1-4e25-b145-c91452ddbf0f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:07.719281 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:37:07.728201 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652ab5aa-2906-4604-8803-d90b3df92ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:07.728909 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ab5aa-2906-4604-8803-d90b3df92ec7 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:07.729218 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ab5aa-2906-4604-8803-d90b3df92ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:07.729497 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ab5aa-2906-4604-8803-d90b3df92ec7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:07.730367 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652ab5aa-2906-4604-8803-d90b3df92ec7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:07.732766 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652ab5aa-2906-4604-8803-d90b3df92ec7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:07.745753 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1361/2728] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:37:07.750136 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004272f4-eb05-4413-97de-5ecb86a6edd5 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:07.761007 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004272f4-eb05-4413-97de-5ecb86a6edd5 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:07.766331 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaeb7cb-08d4-47c8-90f2-491a6e890fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:07.767136 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaeb7cb-08d4-47c8-90f2-491a6e890fb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:07.767420 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaeb7cb-08d4-47c8-90f2-491a6e890fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:07.767735 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaeb7cb-08d4-47c8-90f2-491a6e890fb1 admin admin] PATH_INFO: `/v3/projects/5dc6d1ba779d4da78cace9bf5d3e8821/users/88d6a4c9467b4da9920bf5693a62e425/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:07.769810 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaeb7cb-08d4-47c8-90f2-491a6e890fb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dc6d1ba779d4da78cace9bf5d3e8821, user_id=88d6a4c9467b4da9920bf5693a62e425, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:07.772523 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaeb7cb-08d4-47c8-90f2-491a6e890fb1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:07.783396 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1368/2729] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:37:07 2022] PUT /identity/v3/projects/5dc6d1ba779d4da78cace9bf5d3e8821/users/88d6a4c9467b4da9920bf5693a62e425/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:37:07.793646 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93db7f-fdb0-4672-bc4f-7a5031c49894 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:07.793933 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93db7f-fdb0-4672-bc4f-7a5031c49894 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:07.794207 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93db7f-fdb0-4672-bc4f-7a5031c49894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:07.822770 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-8c93db7f-fdb0-4672-bc4f-7a5031c49894 None None] MFA Rules not processed for user `88d6a4c9467b4da9920bf5693a62e425`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:37:07.823995 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8c93db7f-fdb0-4672-bc4f-7a5031c49894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:37:07.895488 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1362/2730] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:37:07 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:37:07.907979 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaeb7cb-08d4-47c8-90f2-491a6e890fb1 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:07.909974 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7aaeb7cb-08d4-47c8-90f2-491a6e890fb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:37:07.972368 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaeb7cb-08d4-47c8-90f2-491a6e890fb1 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:07.976589 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55bd3b5-c2da-4d15-90ec-d89928df0645 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:07.977327 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55bd3b5-c2da-4d15-90ec-d89928df0645 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:07.977611 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55bd3b5-c2da-4d15-90ec-d89928df0645 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:07.977904 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55bd3b5-c2da-4d15-90ec-d89928df0645 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:07.992586 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55bd3b5-c2da-4d15-90ec-d89928df0645 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:07.994057 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55bd3b5-c2da-4d15-90ec-d89928df0645 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:08.005506 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1369/2731] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:37:07 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:37:08.416741 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c93db7f-fdb0-4672-bc4f-7a5031c49894 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:08.481324 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c93db7f-fdb0-4672-bc4f-7a5031c49894 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:08.486612 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b11cc7a-861c-4f81-9a40-8db87ae256ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:08.487299 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11cc7a-861c-4f81-9a40-8db87ae256ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:08.487578 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11cc7a-861c-4f81-9a40-8db87ae256ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:08.487851 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11cc7a-861c-4f81-9a40-8db87ae256ae admin admin] PATH_INFO: `/v3/users/058557c7cd6f41758868f2df2c1fd165` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:08.489047 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b11cc7a-861c-4f81-9a40-8db87ae256ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=058557c7cd6f41758868f2df2c1fd165)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:08.490996 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b11cc7a-861c-4f81-9a40-8db87ae256ae admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:08.542539 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6b11cc7a-861c-4f81-9a40-8db87ae256ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '058557c7cd6f41758868f2df2c1fd165'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:08.552112 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6b11cc7a-861c-4f81-9a40-8db87ae256ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '058557c7cd6f41758868f2df2c1fd165'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:08.552905 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6b11cc7a-861c-4f81-9a40-8db87ae256ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '058557c7cd6f41758868f2df2c1fd165'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:08.565159 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6b11cc7a-861c-4f81-9a40-8db87ae256ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '058557c7cd6f41758868f2df2c1fd165'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:08.565768 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6b11cc7a-861c-4f81-9a40-8db87ae256ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '058557c7cd6f41758868f2df2c1fd165'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:08.573146 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:37:08.579065 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1363/2732] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:37:08 2022] DELETE /identity/v3/users/058557c7cd6f41758868f2df2c1fd165 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:37:08.584039 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b60354-43af-4f1a-a2b7-16e96745183d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:08.584510 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b60354-43af-4f1a-a2b7-16e96745183d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:08.584605 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b60354-43af-4f1a-a2b7-16e96745183d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:08.604029 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-04b60354-43af-4f1a-a2b7-16e96745183d None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:37:08.605216 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-04b60354-43af-4f1a-a2b7-16e96745183d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:37:08.666493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1370/2733] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:37:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:37:08.676814 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b11cc7a-861c-4f81-9a40-8db87ae256ae admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:08.678261 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6b11cc7a-861c-4f81-9a40-8db87ae256ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:37:08.737647 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b11cc7a-861c-4f81-9a40-8db87ae256ae admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:08.741982 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df36be1-ad9f-4225-be98-8caffb9a339c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:08.742599 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df36be1-ad9f-4225-be98-8caffb9a339c service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:08.742873 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df36be1-ad9f-4225-be98-8caffb9a339c service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:08.743175 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df36be1-ad9f-4225-be98-8caffb9a339c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:08.753882 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df36be1-ad9f-4225-be98-8caffb9a339c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:08.755361 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df36be1-ad9f-4225-be98-8caffb9a339c service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:08.766033 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1364/2734] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:37:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:37:08.774514 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b60354-43af-4f1a-a2b7-16e96745183d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:08.776049 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-04b60354-43af-4f1a-a2b7-16e96745183d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:37:08.788150 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b60354-43af-4f1a-a2b7-16e96745183d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:08.792998 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9524bac-8f07-4754-89da-50454e795a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:08.793645 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9524bac-8f07-4754-89da-50454e795a1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:08.793917 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9524bac-8f07-4754-89da-50454e795a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:08.794184 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9524bac-8f07-4754-89da-50454e795a1e admin admin] PATH_INFO: `/v3/users/1beabcf3cdc143ecb95b765a46c0c60f` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:08.795361 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9524bac-8f07-4754-89da-50454e795a1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1beabcf3cdc143ecb95b765a46c0c60f)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:08.797285 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9524bac-8f07-4754-89da-50454e795a1e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:08.848286 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b9524bac-8f07-4754-89da-50454e795a1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1beabcf3cdc143ecb95b765a46c0c60f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:08.848844 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b9524bac-8f07-4754-89da-50454e795a1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1beabcf3cdc143ecb95b765a46c0c60f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:08.849431 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b9524bac-8f07-4754-89da-50454e795a1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1beabcf3cdc143ecb95b765a46c0c60f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:08.857745 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b9524bac-8f07-4754-89da-50454e795a1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1beabcf3cdc143ecb95b765a46c0c60f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:08.865496 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b9524bac-8f07-4754-89da-50454e795a1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1beabcf3cdc143ecb95b765a46c0c60f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:08.877527 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:37:08.887309 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1371/2735] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:37:08 2022] DELETE /identity/v3/users/1beabcf3cdc143ecb95b765a46c0c60f => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:37:08.892416 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5694c996-1e6b-4ba0-911b-f44783d433c5 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:08.892780 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5694c996-1e6b-4ba0-911b-f44783d433c5 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:08.893160 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5694c996-1e6b-4ba0-911b-f44783d433c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:08.924480 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-5694c996-1e6b-4ba0-911b-f44783d433c5 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:37:08.925719 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5694c996-1e6b-4ba0-911b-f44783d433c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:37:08.987331 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1365/2736] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:37:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:37:08.997253 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9524bac-8f07-4754-89da-50454e795a1e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:08.998701 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b9524bac-8f07-4754-89da-50454e795a1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:37:09.042601 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5694c996-1e6b-4ba0-911b-f44783d433c5 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:09.044188 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5694c996-1e6b-4ba0-911b-f44783d433c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:37:09.058059 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5694c996-1e6b-4ba0-911b-f44783d433c5 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:09.063006 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7545356-951f-4bf1-93a2-4da6eb2bf35e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:09.063796 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7545356-951f-4bf1-93a2-4da6eb2bf35e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:09.064259 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7545356-951f-4bf1-93a2-4da6eb2bf35e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:09.064602 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7545356-951f-4bf1-93a2-4da6eb2bf35e admin admin] PATH_INFO: `/v3/projects/b935a59abdc04b90a26aa301ebfdfe5b` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:09.065855 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7545356-951f-4bf1-93a2-4da6eb2bf35e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b935a59abdc04b90a26aa301ebfdfe5b)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:09.067934 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7545356-951f-4bf1-93a2-4da6eb2bf35e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:09.076464 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9524bac-8f07-4754-89da-50454e795a1e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:09.080428 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a690658-5ab9-46a1-9c0c-c417e422f712 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:09.081057 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a690658-5ab9-46a1-9c0c-c417e422f712 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:09.081339 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a690658-5ab9-46a1-9c0c-c417e422f712 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:09.081612 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a690658-5ab9-46a1-9c0c-c417e422f712 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:09.091238 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a690658-5ab9-46a1-9c0c-c417e422f712 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:09.092678 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a690658-5ab9-46a1-9c0c-c417e422f712 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:09.102463 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1372/2737] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:37:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:37:09.118106 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f7545356-951f-4bf1-93a2-4da6eb2bf35e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b935a59abdc04b90a26aa301ebfdfe5b', 'request_id': 'req-f7545356-951f-4bf1-93a2-4da6eb2bf35e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:09.123611 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f7545356-951f-4bf1-93a2-4da6eb2bf35e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b935a59abdc04b90a26aa301ebfdfe5b', 'request_id': 'req-f7545356-951f-4bf1-93a2-4da6eb2bf35e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:09.133224 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f7545356-951f-4bf1-93a2-4da6eb2bf35e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b935a59abdc04b90a26aa301ebfdfe5b', 'request_id': 'req-f7545356-951f-4bf1-93a2-4da6eb2bf35e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:09.147660 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f7545356-951f-4bf1-93a2-4da6eb2bf35e admin admin] The token cache is being invalidate because project b935a59abdc04b90a26aa301ebfdfe5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:37:09.147948 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f7545356-951f-4bf1-93a2-4da6eb2bf35e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:09.148738 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f7545356-951f-4bf1-93a2-4da6eb2bf35e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:09.154690 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1366/2738] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:37:09 2022] DELETE /identity/v3/projects/b935a59abdc04b90a26aa301ebfdfe5b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:37:09.155544 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a690658-5ab9-46a1-9c0c-c417e422f712 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:09.167733 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a690658-5ab9-46a1-9c0c-c417e422f712 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:09.226526 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d721dfe3-fece-4192-8feb-10de39e20fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:09.227328 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721dfe3-fece-4192-8feb-10de39e20fbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:09.227585 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721dfe3-fece-4192-8feb-10de39e20fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:09.227978 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721dfe3-fece-4192-8feb-10de39e20fbd admin admin] PATH_INFO: `/v3/users/88d6a4c9467b4da9920bf5693a62e425` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:09.229169 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d721dfe3-fece-4192-8feb-10de39e20fbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=88d6a4c9467b4da9920bf5693a62e425)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:09.231123 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d721dfe3-fece-4192-8feb-10de39e20fbd admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:09.286849 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d721dfe3-fece-4192-8feb-10de39e20fbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88d6a4c9467b4da9920bf5693a62e425'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:09.287705 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d721dfe3-fece-4192-8feb-10de39e20fbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88d6a4c9467b4da9920bf5693a62e425'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:09.292375 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d721dfe3-fece-4192-8feb-10de39e20fbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88d6a4c9467b4da9920bf5693a62e425'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:09.301677 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d721dfe3-fece-4192-8feb-10de39e20fbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88d6a4c9467b4da9920bf5693a62e425'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:09.308940 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d721dfe3-fece-4192-8feb-10de39e20fbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88d6a4c9467b4da9920bf5693a62e425'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:09.327737 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1373/2739] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:37:09 2022] DELETE /identity/v3/users/88d6a4c9467b4da9920bf5693a62e425 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:37:09.388142 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7545356-951f-4bf1-93a2-4da6eb2bf35e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:09.389745 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f7545356-951f-4bf1-93a2-4da6eb2bf35e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:37:09.422734 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7545356-951f-4bf1-93a2-4da6eb2bf35e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:09.442013 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:37:09.489378 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b5b5e-e87d-4aca-ba9b-6f024209ee9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:09.490031 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b5b5e-e87d-4aca-ba9b-6f024209ee9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:09.490311 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b5b5e-e87d-4aca-ba9b-6f024209ee9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:09.490608 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b5b5e-e87d-4aca-ba9b-6f024209ee9f admin admin] PATH_INFO: `/v3/projects/dde251794e4b4ea4ae125a8f876cbe3b` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:09.491886 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6b5b5e-e87d-4aca-ba9b-6f024209ee9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=dde251794e4b4ea4ae125a8f876cbe3b)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:09.494027 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6b5b5e-e87d-4aca-ba9b-6f024209ee9f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:09.544988 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9f6b5b5e-e87d-4aca-ba9b-6f024209ee9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dde251794e4b4ea4ae125a8f876cbe3b', 'request_id': 'req-9f6b5b5e-e87d-4aca-ba9b-6f024209ee9f'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:09.552293 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9f6b5b5e-e87d-4aca-ba9b-6f024209ee9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dde251794e4b4ea4ae125a8f876cbe3b', 'request_id': 'req-9f6b5b5e-e87d-4aca-ba9b-6f024209ee9f'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:09.561023 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9f6b5b5e-e87d-4aca-ba9b-6f024209ee9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dde251794e4b4ea4ae125a8f876cbe3b', 'request_id': 'req-9f6b5b5e-e87d-4aca-ba9b-6f024209ee9f'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:09.576942 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9f6b5b5e-e87d-4aca-ba9b-6f024209ee9f admin admin] The token cache is being invalidate because project dde251794e4b4ea4ae125a8f876cbe3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:37:09.577262 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9f6b5b5e-e87d-4aca-ba9b-6f024209ee9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:09.577967 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9f6b5b5e-e87d-4aca-ba9b-6f024209ee9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:09.580318 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1367/2740] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:37:09 2022] DELETE /identity/v3/projects/dde251794e4b4ea4ae125a8f876cbe3b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:37:09.585307 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ddd6d6-94b8-4e16-9782-0c20e6aa09ab None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:09.585829 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ddd6d6-94b8-4e16-9782-0c20e6aa09ab None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:09.585928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ddd6d6-94b8-4e16-9782-0c20e6aa09ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:09.606081 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-60ddd6d6-94b8-4e16-9782-0c20e6aa09ab None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:37:09.607331 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-60ddd6d6-94b8-4e16-9782-0c20e6aa09ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:37:09.617280 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:37:09.624382 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b5b5e-e87d-4aca-ba9b-6f024209ee9f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:09.625879 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9f6b5b5e-e87d-4aca-ba9b-6f024209ee9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:37:09.678611 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1374/2741] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:37:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:37:09.686492 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face6e01-6c8c-4c46-a395-916a021d7ad8 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:09.686803 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face6e01-6c8c-4c46-a395-916a021d7ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:09.687144 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face6e01-6c8c-4c46-a395-916a021d7ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:09.696825 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b5b5e-e87d-4aca-ba9b-6f024209ee9f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:09.701528 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d93539b-1b2a-42a0-ba24-601b0776701b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:09.702263 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93539b-1b2a-42a0-ba24-601b0776701b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:09.702610 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93539b-1b2a-42a0-ba24-601b0776701b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:09.702977 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93539b-1b2a-42a0-ba24-601b0776701b admin admin] PATH_INFO: `/v3/projects/5dc6d1ba779d4da78cace9bf5d3e8821` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:09.704341 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d93539b-1b2a-42a0-ba24-601b0776701b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dc6d1ba779d4da78cace9bf5d3e8821)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:09.706157 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-face6e01-6c8c-4c46-a395-916a021d7ad8 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:37:09.706493 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d93539b-1b2a-42a0-ba24-601b0776701b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:09.707394 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-face6e01-6c8c-4c46-a395-916a021d7ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:37:09.715286 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1375/2742] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:37:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:37:09.719439 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face6e01-6c8c-4c46-a395-916a021d7ad8 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:09.728642 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face6e01-6c8c-4c46-a395-916a021d7ad8 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:09.734455 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6da675-6b10-4453-9154-b3b4e6ba7e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:09.735090 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6da675-6b10-4453-9154-b3b4e6ba7e95 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:09.735366 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6da675-6b10-4453-9154-b3b4e6ba7e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:09.735711 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6da675-6b10-4453-9154-b3b4e6ba7e95 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:09.736540 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6da675-6b10-4453-9154-b3b4e6ba7e95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:37:09.736841 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6da675-6b10-4453-9154-b3b4e6ba7e95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:09.738453 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6da675-6b10-4453-9154-b3b4e6ba7e95 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:09.750435 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1376/2743] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:37:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:37:09.754995 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6da675-6b10-4453-9154-b3b4e6ba7e95 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:09.755172 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3d93539b-1b2a-42a0-ba24-601b0776701b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dc6d1ba779d4da78cace9bf5d3e8821', 'request_id': 'req-3d93539b-1b2a-42a0-ba24-601b0776701b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:09.758811 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:37:09.761959 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3d93539b-1b2a-42a0-ba24-601b0776701b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dc6d1ba779d4da78cace9bf5d3e8821', 'request_id': 'req-3d93539b-1b2a-42a0-ba24-601b0776701b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:09.765857 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6da675-6b10-4453-9154-b3b4e6ba7e95 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:09.770766 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3d93539b-1b2a-42a0-ba24-601b0776701b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dc6d1ba779d4da78cace9bf5d3e8821', 'request_id': 'req-3d93539b-1b2a-42a0-ba24-601b0776701b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:09.783251 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3d93539b-1b2a-42a0-ba24-601b0776701b admin admin] The token cache is being invalidate because project 5dc6d1ba779d4da78cace9bf5d3e8821 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:37:09.783536 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3d93539b-1b2a-42a0-ba24-601b0776701b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:09.784192 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3d93539b-1b2a-42a0-ba24-601b0776701b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:09.785730 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1368/2744] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:37:09 2022] DELETE /identity/v3/projects/5dc6d1ba779d4da78cace9bf5d3e8821 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:37:09.790572 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca43f3-8693-4f23-8abe-6e685fb3e331 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:09.790860 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca43f3-8693-4f23-8abe-6e685fb3e331 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:09.791160 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca43f3-8693-4f23-8abe-6e685fb3e331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:09.811456 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c0ca43f3-8693-4f23-8abe-6e685fb3e331 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:37:09.812672 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c0ca43f3-8693-4f23-8abe-6e685fb3e331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:37:09.827697 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542c262-0263-446f-9b4b-c415e5cad611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:09.828414 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542c262-0263-446f-9b4b-c415e5cad611 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:09.828766 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542c262-0263-446f-9b4b-c415e5cad611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:09.829100 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542c262-0263-446f-9b4b-c415e5cad611 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:09.829990 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b542c262-0263-446f-9b4b-c415e5cad611 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:09.832294 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b542c262-0263-446f-9b4b-c415e5cad611 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:09.863962 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1377/2745] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:37:09 2022] POST /identity/v3/projects => generated 381 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:37:09.868172 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542c262-0263-446f-9b4b-c415e5cad611 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:09.869833 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b542c262-0263-446f-9b4b-c415e5cad611 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:37:09.880205 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1369/2746] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:37:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:37:09.889971 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daee6a7-11c4-4eec-ac0a-bbc2ae423116 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:09.890192 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daee6a7-11c4-4eec-ac0a-bbc2ae423116 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:09.890461 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daee6a7-11c4-4eec-ac0a-bbc2ae423116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:09.913677 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-7daee6a7-11c4-4eec-ac0a-bbc2ae423116 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:37:09.915522 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7daee6a7-11c4-4eec-ac0a-bbc2ae423116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:37:09.937331 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542c262-0263-446f-9b4b-c415e5cad611 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:09.942189 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b4c618-3750-42d0-8adb-b2ddad4bb527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:09.942801 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4c618-3750-42d0-8adb-b2ddad4bb527 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:09.943105 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4c618-3750-42d0-8adb-b2ddad4bb527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:09.943370 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4c618-3750-42d0-8adb-b2ddad4bb527 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:09.944063 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b4c618-3750-42d0-8adb-b2ddad4bb527 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:09.945964 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b4c618-3750-42d0-8adb-b2ddad4bb527 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:09.969946 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1378/2747] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:37:09 2022] POST /identity/v3/users => generated 291 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:37:09.974087 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b4c618-3750-42d0-8adb-b2ddad4bb527 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:09.977513 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1370/2748] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:37:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:37:09.981555 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daee6a7-11c4-4eec-ac0a-bbc2ae423116 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:09.985188 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b4c618-3750-42d0-8adb-b2ddad4bb527 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:09.990484 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daee6a7-11c4-4eec-ac0a-bbc2ae423116 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:09.991349 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6eb60e-a231-4ea3-b656-e7f3e0060b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:09.991962 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6eb60e-a231-4ea3-b656-e7f3e0060b4b admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:09.992238 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6eb60e-a231-4ea3-b656-e7f3e0060b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:09.992502 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6eb60e-a231-4ea3-b656-e7f3e0060b4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:09.993203 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6eb60e-a231-4ea3-b656-e7f3e0060b4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:09.994796 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6eb60e-a231-4ea3-b656-e7f3e0060b4b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:09.995186 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d86934a-407b-4a7e-b3b2-daf24dff3950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:09.995791 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d86934a-407b-4a7e-b3b2-daf24dff3950 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:09.996071 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d86934a-407b-4a7e-b3b2-daf24dff3950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:09.996335 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d86934a-407b-4a7e-b3b2-daf24dff3950 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:09.997040 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d86934a-407b-4a7e-b3b2-daf24dff3950 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:37:09.997315 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d86934a-407b-4a7e-b3b2-daf24dff3950 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:09.998887 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d86934a-407b-4a7e-b3b2-daf24dff3950 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:10.000322 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:37:10.005866 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1379/2749] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:37:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 10:37:10.009707 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6eb60e-a231-4ea3-b656-e7f3e0060b4b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:10.015173 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1371/2750] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:37:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 10:37:10.019723 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d86934a-407b-4a7e-b3b2-daf24dff3950 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:10.025092 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6eb60e-a231-4ea3-b656-e7f3e0060b4b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:10.029343 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d86934a-407b-4a7e-b3b2-daf24dff3950 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:10.030575 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed1b43c-0573-4ec0-a6eb-eb1f6afe580d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:10.031236 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed1b43c-0573-4ec0-a6eb-eb1f6afe580d admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:10.031518 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed1b43c-0573-4ec0-a6eb-eb1f6afe580d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:10.031832 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed1b43c-0573-4ec0-a6eb-eb1f6afe580d admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:10.032529 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed1b43c-0573-4ec0-a6eb-eb1f6afe580d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:10.034139 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed1b43c-0573-4ec0-a6eb-eb1f6afe580d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:10.034625 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60062815-fb1f-4db8-ad25-9111455738ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:10.035267 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60062815-fb1f-4db8-ad25-9111455738ed admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:10.035542 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60062815-fb1f-4db8-ad25-9111455738ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:10.035812 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60062815-fb1f-4db8-ad25-9111455738ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:10.036506 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60062815-fb1f-4db8-ad25-9111455738ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:10.038431 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60062815-fb1f-4db8-ad25-9111455738ed admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:10.045767 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1380/2751] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:37:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:37:10.050173 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed1b43c-0573-4ec0-a6eb-eb1f6afe580d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:10.062154 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed1b43c-0573-4ec0-a6eb-eb1f6afe580d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:10.068060 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11b996d-66fd-4006-8b10-63f11078be2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:10.068332 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1372/2752] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:37:10 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:37:10.068719 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b996d-66fd-4006-8b10-63f11078be2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:10.068992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b996d-66fd-4006-8b10-63f11078be2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:10.069255 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b996d-66fd-4006-8b10-63f11078be2c admin admin] PATH_INFO: `/v3/projects/2bbcb32efae644d8b8ab1c92fb168a7b/users/85904ab7159f486ea9d5075b285148f7/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:10.072425 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60062815-fb1f-4db8-ad25-9111455738ed admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:10.084957 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11b996d-66fd-4006-8b10-63f11078be2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bbcb32efae644d8b8ab1c92fb168a7b, user_id=85904ab7159f486ea9d5075b285148f7, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:10.087721 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11b996d-66fd-4006-8b10-63f11078be2c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:10.102543 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1381/2753] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:37:10 2022] PUT /identity/v3/projects/2bbcb32efae644d8b8ab1c92fb168a7b/users/85904ab7159f486ea9d5075b285148f7/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:37:10.107877 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e68981-284d-464b-8709-506e1beec851 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:10.108189 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e68981-284d-464b-8709-506e1beec851 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:10.108466 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e68981-284d-464b-8709-506e1beec851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:10.123716 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:37:10.128515 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-70e68981-284d-464b-8709-506e1beec851 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:37:10.129864 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-70e68981-284d-464b-8709-506e1beec851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:37:10.141463 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60062815-fb1f-4db8-ad25-9111455738ed admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:10.201302 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26b6455-f97e-43d1-ab76-ed518914e11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:10.201936 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b6455-f97e-43d1-ab76-ed518914e11b admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:10.202212 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b6455-f97e-43d1-ab76-ed518914e11b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:10.202580 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b6455-f97e-43d1-ab76-ed518914e11b admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:10.203278 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26b6455-f97e-43d1-ab76-ed518914e11b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:10.204027 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1382/2754] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:37:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:37:10.205333 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26b6455-f97e-43d1-ab76-ed518914e11b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:10.214127 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e68981-284d-464b-8709-506e1beec851 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:10.216426 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-70e68981-284d-464b-8709-506e1beec851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:37:10.229134 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1373/2755] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:37:10 2022] POST /identity/v3/users => generated 289 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:37:10.234137 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26b6455-f97e-43d1-ab76-ed518914e11b admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:10.243916 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26b6455-f97e-43d1-ab76-ed518914e11b admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:10.248550 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2d4cbf-0648-486c-a754-f4f7dfff90d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:10.249170 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d4cbf-0648-486c-a754-f4f7dfff90d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:10.249459 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d4cbf-0648-486c-a754-f4f7dfff90d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:10.249729 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d4cbf-0648-486c-a754-f4f7dfff90d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:10.250428 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2d4cbf-0648-486c-a754-f4f7dfff90d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:10.252421 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2d4cbf-0648-486c-a754-f4f7dfff90d6 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:10.264057 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1374/2756] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:37:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 10:37:10.268572 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2d4cbf-0648-486c-a754-f4f7dfff90d6 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:10.281536 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2d4cbf-0648-486c-a754-f4f7dfff90d6 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:10.286607 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc6a3ea-d9ed-464b-b6e8-4812c79251fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:10.287372 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6a3ea-d9ed-464b-b6e8-4812c79251fa admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:10.287657 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6a3ea-d9ed-464b-b6e8-4812c79251fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:10.287825 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e68981-284d-464b-8709-506e1beec851 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:10.287933 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6a3ea-d9ed-464b-b6e8-4812c79251fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:10.288629 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc6a3ea-d9ed-464b-b6e8-4812c79251fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:10.290272 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc6a3ea-d9ed-464b-b6e8-4812c79251fa admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:10.291713 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294517f9-f94b-4c10-92c5-e92d766ee394 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:10.292334 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294517f9-f94b-4c10-92c5-e92d766ee394 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:10.292777 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294517f9-f94b-4c10-92c5-e92d766ee394 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:10.292928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294517f9-f94b-4c10-92c5-e92d766ee394 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:10.301554 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1375/2757] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:37:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:37:10.302399 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294517f9-f94b-4c10-92c5-e92d766ee394 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:10.303954 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294517f9-f94b-4c10-92c5-e92d766ee394 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:10.305569 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc6a3ea-d9ed-464b-b6e8-4812c79251fa admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:10.315878 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc6a3ea-d9ed-464b-b6e8-4812c79251fa admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:10.318229 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1383/2758] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:37:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:37:10.321277 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1054e7d3-3ed4-4a07-9b91-eee5c85937c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:10.322024 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054e7d3-3ed4-4a07-9b91-eee5c85937c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:10.322308 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054e7d3-3ed4-4a07-9b91-eee5c85937c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:10.322584 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054e7d3-3ed4-4a07-9b91-eee5c85937c1 admin admin] PATH_INFO: `/v3/projects/8729db611845409b9b536f73a42424e8/users/bfa21d1025444c68a87d2d6560deb062/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:10.338274 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1054e7d3-3ed4-4a07-9b91-eee5c85937c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8729db611845409b9b536f73a42424e8, user_id=bfa21d1025444c68a87d2d6560deb062, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:10.340934 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1054e7d3-3ed4-4a07-9b91-eee5c85937c1 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:10.358599 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1376/2759] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:37:10 2022] PUT /identity/v3/projects/8729db611845409b9b536f73a42424e8/users/bfa21d1025444c68a87d2d6560deb062/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:37:10.369603 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e3d5c-24e5-464a-b272-f603092db638 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:10.370084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e3d5c-24e5-464a-b272-f603092db638 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:10.370362 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e3d5c-24e5-464a-b272-f603092db638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:10.400330 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-613e3d5c-24e5-464a-b272-f603092db638 None None] MFA Rules not processed for user `bfa21d1025444c68a87d2d6560deb062`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:37:10.401603 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-613e3d5c-24e5-464a-b272-f603092db638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:37:10.465325 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:37:10.475077 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1384/2760] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:37:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:37:10.475683 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1054e7d3-3ed4-4a07-9b91-eee5c85937c1 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:10.543907 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1054e7d3-3ed4-4a07-9b91-eee5c85937c1 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:10.548713 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6313c3e5-ab90-4c75-9ef6-c9ed01e88eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:10.549398 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313c3e5-ab90-4c75-9ef6-c9ed01e88eec admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:10.549677 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313c3e5-ab90-4c75-9ef6-c9ed01e88eec admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:10.549966 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313c3e5-ab90-4c75-9ef6-c9ed01e88eec admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:10.550643 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6313c3e5-ab90-4c75-9ef6-c9ed01e88eec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:10.552587 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6313c3e5-ab90-4c75-9ef6-c9ed01e88eec admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:10.581652 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1377/2761] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:37:10 2022] POST /identity/v3/projects => generated 375 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:37:10.585803 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e3d5c-24e5-464a-b272-f603092db638 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:10.596917 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e3d5c-24e5-464a-b272-f603092db638 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:10.683779 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3023d4f9-3a31-4b9d-bbfc-11105431aeee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:10.684708 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3023d4f9-3a31-4b9d-bbfc-11105431aeee admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:10.685234 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3023d4f9-3a31-4b9d-bbfc-11105431aeee admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:10.685724 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3023d4f9-3a31-4b9d-bbfc-11105431aeee admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:10.686968 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3023d4f9-3a31-4b9d-bbfc-11105431aeee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:10.689858 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3023d4f9-3a31-4b9d-bbfc-11105431aeee admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:10.718292 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1385/2762] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:37:10 2022] POST /identity/v3/users => generated 294 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:37:10.723649 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6313c3e5-ab90-4c75-9ef6-c9ed01e88eec admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:10.734160 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6313c3e5-ab90-4c75-9ef6-c9ed01e88eec admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:10.738808 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a5da5f-f4c3-4432-b4f5-e7bda130e264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:10.739462 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5da5f-f4c3-4432-b4f5-e7bda130e264 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:10.739710 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5da5f-f4c3-4432-b4f5-e7bda130e264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:10.739978 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5da5f-f4c3-4432-b4f5-e7bda130e264 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:10.740820 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a5da5f-f4c3-4432-b4f5-e7bda130e264 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:10.743122 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a5da5f-f4c3-4432-b4f5-e7bda130e264 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:10.755646 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1378/2763] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:37:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:37:10.760023 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3023d4f9-3a31-4b9d-bbfc-11105431aeee admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:10.772191 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3023d4f9-3a31-4b9d-bbfc-11105431aeee admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:10.777706 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7750a20-e30c-4adc-8201-833c2a3fec50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:10.778445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7750a20-e30c-4adc-8201-833c2a3fec50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:10.778728 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7750a20-e30c-4adc-8201-833c2a3fec50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:10.779113 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7750a20-e30c-4adc-8201-833c2a3fec50 admin admin] PATH_INFO: `/v3/projects/41b15d86eb4649f0bdb62ca23a8e34ff/users/ee728a71fa3b45e49f210f2a7606d5cb/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:10.794627 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7750a20-e30c-4adc-8201-833c2a3fec50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41b15d86eb4649f0bdb62ca23a8e34ff, user_id=ee728a71fa3b45e49f210f2a7606d5cb, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:10.797300 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7750a20-e30c-4adc-8201-833c2a3fec50 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:10.817645 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1386/2764] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:37:10 2022] PUT /identity/v3/projects/41b15d86eb4649f0bdb62ca23a8e34ff/users/ee728a71fa3b45e49f210f2a7606d5cb/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:37:10.821749 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a5da5f-f4c3-4432-b4f5-e7bda130e264 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:10.832369 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a5da5f-f4c3-4432-b4f5-e7bda130e264 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:10.841888 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:37:10.892188 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f213156-bdef-4bad-85f9-047ca49da4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:10.892814 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f213156-bdef-4bad-85f9-047ca49da4a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:10.893091 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f213156-bdef-4bad-85f9-047ca49da4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:10.893364 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f213156-bdef-4bad-85f9-047ca49da4a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:10.894054 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f213156-bdef-4bad-85f9-047ca49da4a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:10.895687 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f213156-bdef-4bad-85f9-047ca49da4a8 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:10.906609 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1379/2765] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:37:10 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:37:10.911151 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7750a20-e30c-4adc-8201-833c2a3fec50 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:10.922728 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7750a20-e30c-4adc-8201-833c2a3fec50 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:10.927885 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f21ab82-66a0-4b0e-ad89-4726318f7085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:10.928563 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f21ab82-66a0-4b0e-ad89-4726318f7085 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:10.928839 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f21ab82-66a0-4b0e-ad89-4726318f7085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:10.929107 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f21ab82-66a0-4b0e-ad89-4726318f7085 admin admin] PATH_INFO: `/v3/projects/41b15d86eb4649f0bdb62ca23a8e34ff/users/ee728a71fa3b45e49f210f2a7606d5cb/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:10.931138 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f21ab82-66a0-4b0e-ad89-4726318f7085 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41b15d86eb4649f0bdb62ca23a8e34ff, user_id=ee728a71fa3b45e49f210f2a7606d5cb, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:10.933747 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f21ab82-66a0-4b0e-ad89-4726318f7085 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:10.945663 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1387/2766] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:37:10 2022] PUT /identity/v3/projects/41b15d86eb4649f0bdb62ca23a8e34ff/users/ee728a71fa3b45e49f210f2a7606d5cb/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:37:10.957291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9613d2-8c2a-41c9-96ce-7491045154e2 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:10.957640 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9613d2-8c2a-41c9-96ce-7491045154e2 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:10.957966 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9613d2-8c2a-41c9-96ce-7491045154e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:10.998649 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-cc9613d2-8c2a-41c9-96ce-7491045154e2 None None] MFA Rules not processed for user `ee728a71fa3b45e49f210f2a7606d5cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:37:10.999229 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7277db0-03b0-4017-b8c3-ea815ba355cd None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:10.999540 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7277db0-03b0-4017-b8c3-ea815ba355cd None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:10.999793 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7277db0-03b0-4017-b8c3-ea815ba355cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:11.000173 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-cc9613d2-8c2a-41c9-96ce-7491045154e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:37:11.035098 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d7277db0-03b0-4017-b8c3-ea815ba355cd None None] MFA Rules not processed for user `85904ab7159f486ea9d5075b285148f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:37:11.036149 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d7277db0-03b0-4017-b8c3-ea815ba355cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:37:11.086506 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1380/2767] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:37:10 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:37:11.097840 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9613d2-8c2a-41c9-96ce-7491045154e2 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:11.099227 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-cc9613d2-8c2a-41c9-96ce-7491045154e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:37:11.114058 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1388/2768] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:37:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:37:11.121814 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7277db0-03b0-4017-b8c3-ea815ba355cd None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:11.123275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d7277db0-03b0-4017-b8c3-ea815ba355cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:37:11.167939 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9613d2-8c2a-41c9-96ce-7491045154e2 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:11.171621 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26866bd2-5cf7-4935-ade8-920ffa39b628 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:11.172292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26866bd2-5cf7-4935-ade8-920ffa39b628 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:11.172891 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26866bd2-5cf7-4935-ade8-920ffa39b628 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:11.172891 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26866bd2-5cf7-4935-ade8-920ffa39b628 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:11.178421 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:37:11.193881 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7277db0-03b0-4017-b8c3-ea815ba355cd None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:11.197637 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b2603d-bb87-4c33-a2f7-55c3517b891c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:11.198331 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2603d-bb87-4c33-a2f7-55c3517b891c service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:11.198676 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2603d-bb87-4c33-a2f7-55c3517b891c service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:11.199045 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2603d-bb87-4c33-a2f7-55c3517b891c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:11.207790 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b2603d-bb87-4c33-a2f7-55c3517b891c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:11.209304 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b2603d-bb87-4c33-a2f7-55c3517b891c service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:11.218923 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1389/2769] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:37:11 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:37:11.227174 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:37:11.238833 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26866bd2-5cf7-4935-ade8-920ffa39b628 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:11.240305 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26866bd2-5cf7-4935-ade8-920ffa39b628 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:11.249506 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1381/2770] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:37:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 10:37:12.660956 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b294bc2-ee29-4ec1-9cf5-e8f51a2639c8 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:12.661584 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b294bc2-ee29-4ec1-9cf5-e8f51a2639c8 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:12.661584 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b294bc2-ee29-4ec1-9cf5-e8f51a2639c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:12.682657 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8b294bc2-ee29-4ec1-9cf5-e8f51a2639c8 None None] MFA Rules not processed for user `85904ab7159f486ea9d5075b285148f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:37:12.683901 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8b294bc2-ee29-4ec1-9cf5-e8f51a2639c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:37:12.692054 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1390/2771] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:37:12 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:37:18.036660 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26866bd2-5cf7-4935-ade8-920ffa39b628 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:18.046300 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26866bd2-5cf7-4935-ade8-920ffa39b628 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:18.050775 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecdcab7-a3ff-47c7-a3e9-0d6f6ead253c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:18.051449 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecdcab7-a3ff-47c7-a3e9-0d6f6ead253c service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:18.051751 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecdcab7-a3ff-47c7-a3e9-0d6f6ead253c service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:18.051994 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecdcab7-a3ff-47c7-a3e9-0d6f6ead253c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:18.061671 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecdcab7-a3ff-47c7-a3e9-0d6f6ead253c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:18.063264 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecdcab7-a3ff-47c7-a3e9-0d6f6ead253c service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:18.074521 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1382/2772] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:37:18 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:37:18.157105 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b294bc2-ee29-4ec1-9cf5-e8f51a2639c8 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:18.218628 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b294bc2-ee29-4ec1-9cf5-e8f51a2639c8 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:18.222626 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117c6c98-f39e-40b0-be3f-9e79298aa32a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:18.223297 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c6c98-f39e-40b0-be3f-9e79298aa32a service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:18.223559 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c6c98-f39e-40b0-be3f-9e79298aa32a service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:18.223863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c6c98-f39e-40b0-be3f-9e79298aa32a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:18.225707 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-117c6c98-f39e-40b0-be3f-9e79298aa32a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:37:18.236219 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117c6c98-f39e-40b0-be3f-9e79298aa32a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:18.237774 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117c6c98-f39e-40b0-be3f-9e79298aa32a service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:18.246038 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1391/2773] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:37:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:37:18.830606 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecdcab7-a3ff-47c7-a3e9-0d6f6ead253c service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:18.832709 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9ecdcab7-a3ff-47c7-a3e9-0d6f6ead253c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:37:18.915488 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecdcab7-a3ff-47c7-a3e9-0d6f6ead253c service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:18.921127 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3277ed-7509-4acc-bd85-c991d8c05149 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:18.921760 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3277ed-7509-4acc-bd85-c991d8c05149 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:18.922052 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3277ed-7509-4acc-bd85-c991d8c05149 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:18.922319 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3277ed-7509-4acc-bd85-c991d8c05149 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:18.924150 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-bb3277ed-7509-4acc-bd85-c991d8c05149 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:37:18.937442 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3277ed-7509-4acc-bd85-c991d8c05149 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:18.939069 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3277ed-7509-4acc-bd85-c991d8c05149 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:18.950180 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1383/2774] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:37:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:37:28.242638 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=37 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:37:28.945787 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=45 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:37:31.598776 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117c6c98-f39e-40b0-be3f-9e79298aa32a service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:31.600606 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-117c6c98-f39e-40b0-be3f-9e79298aa32a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:37:31.622084 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117c6c98-f39e-40b0-be3f-9e79298aa32a service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:31.628723 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b616326-7673-43e8-be61-e58b6e94d528 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:31.629690 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b616326-7673-43e8-be61-e58b6e94d528 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:31.630235 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b616326-7673-43e8-be61-e58b6e94d528 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:31.630775 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b616326-7673-43e8-be61-e58b6e94d528 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:31.633008 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6b616326-7673-43e8-be61-e58b6e94d528 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:37:31.650844 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b616326-7673-43e8-be61-e58b6e94d528 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:31.652588 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b616326-7673-43e8-be61-e58b6e94d528 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:31.668554 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1392/2775] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:37:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 10:37:41.662764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=6 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:37:51.540276 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3277ed-7509-4acc-bd85-c991d8c05149 service nova] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:51.611191 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3277ed-7509-4acc-bd85-c991d8c05149 service nova] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:51.616687 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbfc80b-c275-4181-b6c3-f093c9ddd9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:51.617379 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbfc80b-c275-4181-b6c3-f093c9ddd9aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:51.617652 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbfc80b-c275-4181-b6c3-f093c9ddd9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:51.617959 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbfc80b-c275-4181-b6c3-f093c9ddd9aa admin admin] PATH_INFO: `/v3/users/bfa21d1025444c68a87d2d6560deb062` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:51.619267 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbfc80b-c275-4181-b6c3-f093c9ddd9aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfa21d1025444c68a87d2d6560deb062)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:51.621174 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbfc80b-c275-4181-b6c3-f093c9ddd9aa admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:51.676014 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-fdbfc80b-c275-4181-b6c3-f093c9ddd9aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfa21d1025444c68a87d2d6560deb062'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:51.685940 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-fdbfc80b-c275-4181-b6c3-f093c9ddd9aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfa21d1025444c68a87d2d6560deb062'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:51.687668 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-fdbfc80b-c275-4181-b6c3-f093c9ddd9aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfa21d1025444c68a87d2d6560deb062'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:51.707339 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-fdbfc80b-c275-4181-b6c3-f093c9ddd9aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfa21d1025444c68a87d2d6560deb062'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:51.708921 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-fdbfc80b-c275-4181-b6c3-f093c9ddd9aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfa21d1025444c68a87d2d6560deb062'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:51.723921 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1384/2776] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:37:51 2022] DELETE /identity/v3/users/bfa21d1025444c68a87d2d6560deb062 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:37:51.728630 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2cd48d-fd2c-49e1-ae4d-e422e07e0f48 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:51.729124 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2cd48d-fd2c-49e1-ae4d-e422e07e0f48 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:51.729288 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2cd48d-fd2c-49e1-ae4d-e422e07e0f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:51.749850 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-fb2cd48d-fd2c-49e1-ae4d-e422e07e0f48 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:37:51.751138 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fb2cd48d-fd2c-49e1-ae4d-e422e07e0f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:37:51.819969 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1393/2777] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:37:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:37:51.832205 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbfc80b-c275-4181-b6c3-f093c9ddd9aa admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:51.833879 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fdbfc80b-c275-4181-b6c3-f093c9ddd9aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:37:51.894428 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbfc80b-c275-4181-b6c3-f093c9ddd9aa admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:51.898565 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b43318-9619-4e00-9728-ad90ef821f38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:51.899141 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b43318-9619-4e00-9728-ad90ef821f38 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:51.899425 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b43318-9619-4e00-9728-ad90ef821f38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:51.899698 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b43318-9619-4e00-9728-ad90ef821f38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:51.910101 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b43318-9619-4e00-9728-ad90ef821f38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:51.911614 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b43318-9619-4e00-9728-ad90ef821f38 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:51.921481 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1385/2778] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:37:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:37:52.218409 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2cd48d-fd2c-49e1-ae4d-e422e07e0f48 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:52.219994 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fb2cd48d-fd2c-49e1-ae4d-e422e07e0f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:37:52.232410 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2cd48d-fd2c-49e1-ae4d-e422e07e0f48 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:52.237134 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb814c83-a415-45e5-bc78-72d45561fdd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:52.237772 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb814c83-a415-45e5-bc78-72d45561fdd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:52.238066 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb814c83-a415-45e5-bc78-72d45561fdd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:52.238322 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb814c83-a415-45e5-bc78-72d45561fdd3 admin admin] PATH_INFO: `/v3/projects/8729db611845409b9b536f73a42424e8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:52.239673 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb814c83-a415-45e5-bc78-72d45561fdd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8729db611845409b9b536f73a42424e8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:52.241532 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb814c83-a415-45e5-bc78-72d45561fdd3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:52.291995 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cb814c83-a415-45e5-bc78-72d45561fdd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8729db611845409b9b536f73a42424e8', 'request_id': 'req-cb814c83-a415-45e5-bc78-72d45561fdd3'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:52.301017 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cb814c83-a415-45e5-bc78-72d45561fdd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8729db611845409b9b536f73a42424e8', 'request_id': 'req-cb814c83-a415-45e5-bc78-72d45561fdd3'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:52.307391 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cb814c83-a415-45e5-bc78-72d45561fdd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8729db611845409b9b536f73a42424e8', 'request_id': 'req-cb814c83-a415-45e5-bc78-72d45561fdd3'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:52.322527 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cb814c83-a415-45e5-bc78-72d45561fdd3 admin admin] The token cache is being invalidate because project 8729db611845409b9b536f73a42424e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:37:52.322824 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cb814c83-a415-45e5-bc78-72d45561fdd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:52.323612 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cb814c83-a415-45e5-bc78-72d45561fdd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:52.325272 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1394/2779] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:37:52 2022] DELETE /identity/v3/projects/8729db611845409b9b536f73a42424e8 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:37:52.329449 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b43318-9619-4e00-9728-ad90ef821f38 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:52.342775 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b43318-9619-4e00-9728-ad90ef821f38 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:52.361183 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:37:52.403086 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677390da-21e5-4a4a-bcde-2144774f3e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:52.404148 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677390da-21e5-4a4a-bcde-2144774f3e54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:52.404441 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677390da-21e5-4a4a-bcde-2144774f3e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:52.404719 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677390da-21e5-4a4a-bcde-2144774f3e54 admin admin] PATH_INFO: `/v3/users/ee728a71fa3b45e49f210f2a7606d5cb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:52.405932 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677390da-21e5-4a4a-bcde-2144774f3e54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee728a71fa3b45e49f210f2a7606d5cb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:52.408044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677390da-21e5-4a4a-bcde-2144774f3e54 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:52.458586 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-677390da-21e5-4a4a-bcde-2144774f3e54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee728a71fa3b45e49f210f2a7606d5cb'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:52.467218 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-677390da-21e5-4a4a-bcde-2144774f3e54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee728a71fa3b45e49f210f2a7606d5cb'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:52.467870 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-677390da-21e5-4a4a-bcde-2144774f3e54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee728a71fa3b45e49f210f2a7606d5cb'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:52.479596 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-677390da-21e5-4a4a-bcde-2144774f3e54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee728a71fa3b45e49f210f2a7606d5cb'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:52.480264 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-677390da-21e5-4a4a-bcde-2144774f3e54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee728a71fa3b45e49f210f2a7606d5cb'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:52.492883 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1386/2780] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:37:52 2022] DELETE /identity/v3/users/ee728a71fa3b45e49f210f2a7606d5cb => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:37:52.772012 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb814c83-a415-45e5-bc78-72d45561fdd3 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:52.773528 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cb814c83-a415-45e5-bc78-72d45561fdd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:37:52.787979 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb814c83-a415-45e5-bc78-72d45561fdd3 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:52.855321 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca93a98c-5ac9-4788-b6a2-1374cecbe91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:52.855900 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93a98c-5ac9-4788-b6a2-1374cecbe91f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:52.856177 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93a98c-5ac9-4788-b6a2-1374cecbe91f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:52.856671 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93a98c-5ac9-4788-b6a2-1374cecbe91f admin admin] PATH_INFO: `/v3/projects/41b15d86eb4649f0bdb62ca23a8e34ff` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:52.857960 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca93a98c-5ac9-4788-b6a2-1374cecbe91f admin admin] RBAC: Authorizing `identity:delete_project(project_id=41b15d86eb4649f0bdb62ca23a8e34ff)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:52.859949 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca93a98c-5ac9-4788-b6a2-1374cecbe91f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:52.904823 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:37:52.920790 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ca93a98c-5ac9-4788-b6a2-1374cecbe91f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41b15d86eb4649f0bdb62ca23a8e34ff', 'request_id': 'req-ca93a98c-5ac9-4788-b6a2-1374cecbe91f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:52.929572 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ca93a98c-5ac9-4788-b6a2-1374cecbe91f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41b15d86eb4649f0bdb62ca23a8e34ff', 'request_id': 'req-ca93a98c-5ac9-4788-b6a2-1374cecbe91f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:52.935279 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ca93a98c-5ac9-4788-b6a2-1374cecbe91f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41b15d86eb4649f0bdb62ca23a8e34ff', 'request_id': 'req-ca93a98c-5ac9-4788-b6a2-1374cecbe91f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:52.946425 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ca93a98c-5ac9-4788-b6a2-1374cecbe91f admin admin] The token cache is being invalidate because project 41b15d86eb4649f0bdb62ca23a8e34ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:37:52.946713 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ca93a98c-5ac9-4788-b6a2-1374cecbe91f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:52.947475 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ca93a98c-5ac9-4788-b6a2-1374cecbe91f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:37:52.949088 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1395/2781] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:37:52 2022] DELETE /identity/v3/projects/41b15d86eb4649f0bdb62ca23a8e34ff => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:37:52.955609 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad24c6-7aed-4d80-90fe-0f71138ac1a4 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:52.956131 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad24c6-7aed-4d80-90fe-0f71138ac1a4 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:52.956131 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad24c6-7aed-4d80-90fe-0f71138ac1a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:52.976853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-acad24c6-7aed-4d80-90fe-0f71138ac1a4 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:37:52.978067 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-acad24c6-7aed-4d80-90fe-0f71138ac1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:37:53.048281 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1387/2782] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:37:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:37:53.056267 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26c86e-f1f5-4735-a751-a3a0cdccf0ef None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:53.056553 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26c86e-f1f5-4735-a751-a3a0cdccf0ef None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:53.056829 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26c86e-f1f5-4735-a751-a3a0cdccf0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:53.077634 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ca26c86e-f1f5-4735-a751-a3a0cdccf0ef None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:37:53.078839 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ca26c86e-f1f5-4735-a751-a3a0cdccf0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:37:53.087334 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1396/2783] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:37:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:37:53.091931 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad24c6-7aed-4d80-90fe-0f71138ac1a4 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:53.101869 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad24c6-7aed-4d80-90fe-0f71138ac1a4 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:53.106640 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8130b72d-7b01-4ae8-9b6c-6528723af92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:53.107322 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130b72d-7b01-4ae8-9b6c-6528723af92e admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:53.107601 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130b72d-7b01-4ae8-9b6c-6528723af92e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:53.107869 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130b72d-7b01-4ae8-9b6c-6528723af92e admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:53.108563 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8130b72d-7b01-4ae8-9b6c-6528723af92e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:37:53.108864 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8130b72d-7b01-4ae8-9b6c-6528723af92e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:53.110458 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8130b72d-7b01-4ae8-9b6c-6528723af92e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:53.122192 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1388/2784] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:37:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:37:53.127470 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca26c86e-f1f5-4735-a751-a3a0cdccf0ef None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:53.142705 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca26c86e-f1f5-4735-a751-a3a0cdccf0ef None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:53.151702 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b047e67d-d5ac-4eab-bea1-9feb5d7e5dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:53.152594 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047e67d-d5ac-4eab-bea1-9feb5d7e5dad admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:53.153078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047e67d-d5ac-4eab-bea1-9feb5d7e5dad admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:53.153422 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047e67d-d5ac-4eab-bea1-9feb5d7e5dad admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:53.154205 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b047e67d-d5ac-4eab-bea1-9feb5d7e5dad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:53.156235 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b047e67d-d5ac-4eab-bea1-9feb5d7e5dad admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:53.188124 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1397/2785] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:37:53 2022] POST /identity/v3/projects => generated 407 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:37:53.192374 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8130b72d-7b01-4ae8-9b6c-6528723af92e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:53.202474 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8130b72d-7b01-4ae8-9b6c-6528723af92e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:53.230409 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:37:53.261311 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b97cd2-9bd4-4380-b27c-3ea8a4e1aa0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:53.261935 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b97cd2-9bd4-4380-b27c-3ea8a4e1aa0b admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:53.262220 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b97cd2-9bd4-4380-b27c-3ea8a4e1aa0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:53.262493 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b97cd2-9bd4-4380-b27c-3ea8a4e1aa0b admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:53.263274 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b97cd2-9bd4-4380-b27c-3ea8a4e1aa0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:53.265200 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b97cd2-9bd4-4380-b27c-3ea8a4e1aa0b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:53.288986 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1389/2786] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:37:53 2022] POST /identity/v3/users => generated 304 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:37:53.293297 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b047e67d-d5ac-4eab-bea1-9feb5d7e5dad admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:53.303832 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b047e67d-d5ac-4eab-bea1-9feb5d7e5dad admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:53.308653 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6164c8-c6d4-46d9-b846-badf847e7166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:53.309269 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6164c8-c6d4-46d9-b846-badf847e7166 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:53.309554 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6164c8-c6d4-46d9-b846-badf847e7166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:53.309960 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6164c8-c6d4-46d9-b846-badf847e7166 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:53.310674 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6164c8-c6d4-46d9-b846-badf847e7166 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:53.312327 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6164c8-c6d4-46d9-b846-badf847e7166 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:53.323482 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1398/2787] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:37:53.327707 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b97cd2-9bd4-4380-b27c-3ea8a4e1aa0b admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:53.338189 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b97cd2-9bd4-4380-b27c-3ea8a4e1aa0b admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:53.343462 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9265b08-d15f-41d5-9f01-d6d9dd5ed3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:53.344077 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9265b08-d15f-41d5-9f01-d6d9dd5ed3d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:53.344333 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9265b08-d15f-41d5-9f01-d6d9dd5ed3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:53.344599 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9265b08-d15f-41d5-9f01-d6d9dd5ed3d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:53.345292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9265b08-d15f-41d5-9f01-d6d9dd5ed3d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:53.346887 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9265b08-d15f-41d5-9f01-d6d9dd5ed3d6 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:53.357864 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1390/2788] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:37:53.362009 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6164c8-c6d4-46d9-b846-badf847e7166 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:53.372571 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6164c8-c6d4-46d9-b846-badf847e7166 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:53.377548 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efc134a-f8f6-4eb5-b3c7-a190428d72ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:53.378211 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc134a-f8f6-4eb5-b3c7-a190428d72ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:53.378485 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc134a-f8f6-4eb5-b3c7-a190428d72ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:53.378764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc134a-f8f6-4eb5-b3c7-a190428d72ff admin admin] PATH_INFO: `/v3/projects/a61cc72410e24b139a108b39673ad3b1/users/fe59186f64b94a2ca4325ec0a3fd4f18/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:53.394785 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efc134a-f8f6-4eb5-b3c7-a190428d72ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=a61cc72410e24b139a108b39673ad3b1, user_id=fe59186f64b94a2ca4325ec0a3fd4f18, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:53.397458 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efc134a-f8f6-4eb5-b3c7-a190428d72ff admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:53.412900 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1399/2789] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:37:53 2022] PUT /identity/v3/projects/a61cc72410e24b139a108b39673ad3b1/users/fe59186f64b94a2ca4325ec0a3fd4f18/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:37:53.423393 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdccfb50-fdab-4d3c-adf4-fc74a1c2b885 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:53.423643 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdccfb50-fdab-4d3c-adf4-fc74a1c2b885 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:53.423944 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdccfb50-fdab-4d3c-adf4-fc74a1c2b885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:53.453343 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-fdccfb50-fdab-4d3c-adf4-fc74a1c2b885 None None] MFA Rules not processed for user `fe59186f64b94a2ca4325ec0a3fd4f18`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:37:53.454549 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fdccfb50-fdab-4d3c-adf4-fc74a1c2b885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:37:53.525063 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1391/2790] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:37:53 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:37:53.529870 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efc134a-f8f6-4eb5-b3c7-a190428d72ff admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:53.593605 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efc134a-f8f6-4eb5-b3c7-a190428d72ff admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:53.602042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf1035b-b36a-41f9-b4e4-b2398a244ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:53.602861 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1035b-b36a-41f9-b4e4-b2398a244ad8 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:53.603245 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1035b-b36a-41f9-b4e4-b2398a244ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:53.603512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1035b-b36a-41f9-b4e4-b2398a244ad8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:53.604449 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf1035b-b36a-41f9-b4e4-b2398a244ad8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:53.607238 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf1035b-b36a-41f9-b4e4-b2398a244ad8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:53.637937 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1400/2791] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:37:53 2022] POST /identity/v3/projects => generated 409 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:37:53.644543 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdccfb50-fdab-4d3c-adf4-fc74a1c2b885 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:53.660386 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdccfb50-fdab-4d3c-adf4-fc74a1c2b885 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:53.717133 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce14c99-fb0b-4084-b631-09429539fa36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:53.717740 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce14c99-fb0b-4084-b631-09429539fa36 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:53.718021 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce14c99-fb0b-4084-b631-09429539fa36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:53.718292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce14c99-fb0b-4084-b631-09429539fa36 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:53.718940 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce14c99-fb0b-4084-b631-09429539fa36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:53.720869 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce14c99-fb0b-4084-b631-09429539fa36 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:53.743472 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1392/2792] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:37:53 2022] POST /identity/v3/users => generated 311 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:37:53.747741 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf1035b-b36a-41f9-b4e4-b2398a244ad8 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:53.758021 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf1035b-b36a-41f9-b4e4-b2398a244ad8 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:53.763176 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fc78e1-3eb4-4006-b3de-8d1bdebf0d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:53.763972 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fc78e1-3eb4-4006-b3de-8d1bdebf0d3e admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:53.764253 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fc78e1-3eb4-4006-b3de-8d1bdebf0d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:53.764485 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fc78e1-3eb4-4006-b3de-8d1bdebf0d3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:53.765181 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fc78e1-3eb4-4006-b3de-8d1bdebf0d3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:53.766763 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fc78e1-3eb4-4006-b3de-8d1bdebf0d3e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:53.778372 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1401/2793] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:37:53.782814 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce14c99-fb0b-4084-b631-09429539fa36 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:53.793515 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce14c99-fb0b-4084-b631-09429539fa36 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:53.799630 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babf4510-cd72-45cc-8ca3-b65c2c078978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:53.800319 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf4510-cd72-45cc-8ca3-b65c2c078978 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:53.800596 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf4510-cd72-45cc-8ca3-b65c2c078978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:53.800872 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf4510-cd72-45cc-8ca3-b65c2c078978 admin admin] PATH_INFO: `/v3/projects/530ce79d58484c4c98712e932643ddb0/users/e9779ffdc7ea45f7b43e2915f467d6af/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:53.817025 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babf4510-cd72-45cc-8ca3-b65c2c078978 admin admin] RBAC: Authorizing `identity:create_grant(project_id=530ce79d58484c4c98712e932643ddb0, user_id=e9779ffdc7ea45f7b43e2915f467d6af, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:53.819725 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babf4510-cd72-45cc-8ca3-b65c2c078978 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:53.837772 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1393/2794] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:37:53 2022] PUT /identity/v3/projects/530ce79d58484c4c98712e932643ddb0/users/e9779ffdc7ea45f7b43e2915f467d6af/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:37:53.842242 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fc78e1-3eb4-4006-b3de-8d1bdebf0d3e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:53.852484 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fc78e1-3eb4-4006-b3de-8d1bdebf0d3e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:53.872839 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:37:53.914004 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c2e64-c59c-46c4-890e-8000ebf1cda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:53.914635 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c2e64-c59c-46c4-890e-8000ebf1cda6 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:53.914937 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c2e64-c59c-46c4-890e-8000ebf1cda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:53.915245 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c2e64-c59c-46c4-890e-8000ebf1cda6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:53.915922 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43c2e64-c59c-46c4-890e-8000ebf1cda6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:53.917535 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43c2e64-c59c-46c4-890e-8000ebf1cda6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:53.927982 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1402/2795] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:37:53.932430 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babf4510-cd72-45cc-8ca3-b65c2c078978 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:53.938143 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:37:53.944996 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babf4510-cd72-45cc-8ca3-b65c2c078978 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:53.950662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e804852-ce43-47c6-abde-34e600564717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:53.951366 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e804852-ce43-47c6-abde-34e600564717 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:53.951643 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e804852-ce43-47c6-abde-34e600564717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:53.951911 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e804852-ce43-47c6-abde-34e600564717 admin admin] PATH_INFO: `/v3/projects/530ce79d58484c4c98712e932643ddb0/users/e9779ffdc7ea45f7b43e2915f467d6af/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:53.953990 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e804852-ce43-47c6-abde-34e600564717 admin admin] RBAC: Authorizing `identity:create_grant(project_id=530ce79d58484c4c98712e932643ddb0, user_id=e9779ffdc7ea45f7b43e2915f467d6af, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:53.956680 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e804852-ce43-47c6-abde-34e600564717 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:53.968049 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1394/2796] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:37:53 2022] PUT /identity/v3/projects/530ce79d58484c4c98712e932643ddb0/users/e9779ffdc7ea45f7b43e2915f467d6af/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:37:53.979205 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c61cf-c437-4d88-ac17-81275b830de8 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:53.979512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c61cf-c437-4d88-ac17-81275b830de8 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:53.979812 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c61cf-c437-4d88-ac17-81275b830de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:54.017024 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-680c61cf-c437-4d88-ac17-81275b830de8 None None] MFA Rules not processed for user `e9779ffdc7ea45f7b43e2915f467d6af`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:37:54.018256 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-680c61cf-c437-4d88-ac17-81275b830de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:37:54.087543 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1403/2797] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:37:53 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:37:54.099890 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e804852-ce43-47c6-abde-34e600564717 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:54.102674 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2e804852-ce43-47c6-abde-34e600564717 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:37:54.166180 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e804852-ce43-47c6-abde-34e600564717 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:54.170341 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfa5aa8-1127-455f-892d-2b216e02f9ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:54.170996 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfa5aa8-1127-455f-892d-2b216e02f9ea service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:54.171424 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfa5aa8-1127-455f-892d-2b216e02f9ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:54.171699 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfa5aa8-1127-455f-892d-2b216e02f9ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:54.181542 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfa5aa8-1127-455f-892d-2b216e02f9ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:54.183051 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfa5aa8-1127-455f-892d-2b216e02f9ea service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:54.193277 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1395/2798] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:37:54 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:37:54.355008 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c61cf-c437-4d88-ac17-81275b830de8 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:37:54.366653 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c61cf-c437-4d88-ac17-81275b830de8 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:37:54.371271 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e27f0-8ea5-4f8d-97cd-88810930e636 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:37:54.371897 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e27f0-8ea5-4f8d-97cd-88810930e636 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:37:54.372190 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e27f0-8ea5-4f8d-97cd-88810930e636 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:37:54.372452 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e27f0-8ea5-4f8d-97cd-88810930e636 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:37:54.434477 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33e27f0-8ea5-4f8d-97cd-88810930e636 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:37:54.436066 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33e27f0-8ea5-4f8d-97cd-88810930e636 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:37:54.445212 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1404/2799] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:37:54 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:38:04.188675 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:38:04.441336 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=80 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:38:06.051567 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfa5aa8-1127-455f-892d-2b216e02f9ea service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:38:06.117271 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfa5aa8-1127-455f-892d-2b216e02f9ea service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:38:06.122182 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff3522-6307-4f5f-8599-40bdf3bafafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:38:06.122817 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff3522-6307-4f5f-8599-40bdf3bafafc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:06.123134 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff3522-6307-4f5f-8599-40bdf3bafafc admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:06.123409 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff3522-6307-4f5f-8599-40bdf3bafafc admin admin] PATH_INFO: `/v3/users/fe59186f64b94a2ca4325ec0a3fd4f18` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:06.124768 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ff3522-6307-4f5f-8599-40bdf3bafafc admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe59186f64b94a2ca4325ec0a3fd4f18)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:38:06.127143 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ff3522-6307-4f5f-8599-40bdf3bafafc admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:38:06.184298 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d5ff3522-6307-4f5f-8599-40bdf3bafafc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe59186f64b94a2ca4325ec0a3fd4f18'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:06.193521 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d5ff3522-6307-4f5f-8599-40bdf3bafafc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe59186f64b94a2ca4325ec0a3fd4f18'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:06.194152 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d5ff3522-6307-4f5f-8599-40bdf3bafafc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe59186f64b94a2ca4325ec0a3fd4f18'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:06.206521 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d5ff3522-6307-4f5f-8599-40bdf3bafafc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe59186f64b94a2ca4325ec0a3fd4f18'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:06.207176 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d5ff3522-6307-4f5f-8599-40bdf3bafafc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe59186f64b94a2ca4325ec0a3fd4f18'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:06.219735 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1396/2800] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:38:06 2022] DELETE /identity/v3/users/fe59186f64b94a2ca4325ec0a3fd4f18 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:38:06.224464 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e9c07-3fe9-49a2-9e60-779d57f7a292 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:06.224985 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e9c07-3fe9-49a2-9e60-779d57f7a292 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:06.225098 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e9c07-3fe9-49a2-9e60-779d57f7a292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:06.244580 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ee2e9c07-3fe9-49a2-9e60-779d57f7a292 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:38:06.245781 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ee2e9c07-3fe9-49a2-9e60-779d57f7a292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:38:06.310381 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1405/2801] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:38:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:38:06.320991 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff3522-6307-4f5f-8599-40bdf3bafafc admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:38:06.322700 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d5ff3522-6307-4f5f-8599-40bdf3bafafc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:38:06.382615 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff3522-6307-4f5f-8599-40bdf3bafafc admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:38:06.386936 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8c1070-67de-4ed3-beca-9c23de5ac9ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:38:06.387565 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c1070-67de-4ed3-beca-9c23de5ac9ca service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:06.387837 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c1070-67de-4ed3-beca-9c23de5ac9ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:06.388104 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c1070-67de-4ed3-beca-9c23de5ac9ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:06.403232 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8c1070-67de-4ed3-beca-9c23de5ac9ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:38:06.405417 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8c1070-67de-4ed3-beca-9c23de5ac9ca service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:38:06.420399 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1397/2802] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:38:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:38:06.694144 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e9c07-3fe9-49a2-9e60-779d57f7a292 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:38:06.695612 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ee2e9c07-3fe9-49a2-9e60-779d57f7a292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:38:06.709255 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e9c07-3fe9-49a2-9e60-779d57f7a292 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:38:06.715182 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6ad137-c618-4097-ac6d-06d7d36a9ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:38:06.715846 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6ad137-c618-4097-ac6d-06d7d36a9ab2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:06.716163 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6ad137-c618-4097-ac6d-06d7d36a9ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:06.717056 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6ad137-c618-4097-ac6d-06d7d36a9ab2 admin admin] PATH_INFO: `/v3/projects/a61cc72410e24b139a108b39673ad3b1` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:06.718797 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6ad137-c618-4097-ac6d-06d7d36a9ab2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a61cc72410e24b139a108b39673ad3b1)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:38:06.720789 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6ad137-c618-4097-ac6d-06d7d36a9ab2 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:38:06.769026 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4a6ad137-c618-4097-ac6d-06d7d36a9ab2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a61cc72410e24b139a108b39673ad3b1', 'request_id': 'req-4a6ad137-c618-4097-ac6d-06d7d36a9ab2'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:06.777801 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4a6ad137-c618-4097-ac6d-06d7d36a9ab2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a61cc72410e24b139a108b39673ad3b1', 'request_id': 'req-4a6ad137-c618-4097-ac6d-06d7d36a9ab2'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:06.783523 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4a6ad137-c618-4097-ac6d-06d7d36a9ab2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a61cc72410e24b139a108b39673ad3b1', 'request_id': 'req-4a6ad137-c618-4097-ac6d-06d7d36a9ab2'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:06.794686 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4a6ad137-c618-4097-ac6d-06d7d36a9ab2 admin admin] The token cache is being invalidate because project a61cc72410e24b139a108b39673ad3b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:38:06.795009 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4a6ad137-c618-4097-ac6d-06d7d36a9ab2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:06.795717 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4a6ad137-c618-4097-ac6d-06d7d36a9ab2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:06.797334 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1406/2803] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:38:06 2022] DELETE /identity/v3/projects/a61cc72410e24b139a108b39673ad3b1 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:38:06.801494 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8c1070-67de-4ed3-beca-9c23de5ac9ca service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:38:06.811613 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8c1070-67de-4ed3-beca-9c23de5ac9ca service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:38:06.831426 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:38:06.877768 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02df601-a348-4232-a3ec-a01f1363e764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:38:06.878408 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02df601-a348-4232-a3ec-a01f1363e764 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:06.878686 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02df601-a348-4232-a3ec-a01f1363e764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:06.878979 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02df601-a348-4232-a3ec-a01f1363e764 admin admin] PATH_INFO: `/v3/users/e9779ffdc7ea45f7b43e2915f467d6af` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:06.880165 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02df601-a348-4232-a3ec-a01f1363e764 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9779ffdc7ea45f7b43e2915f467d6af)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:38:06.882087 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02df601-a348-4232-a3ec-a01f1363e764 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:38:06.935249 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-b02df601-a348-4232-a3ec-a01f1363e764 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9779ffdc7ea45f7b43e2915f467d6af'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:06.946474 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-b02df601-a348-4232-a3ec-a01f1363e764 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9779ffdc7ea45f7b43e2915f467d6af'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:06.947267 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-b02df601-a348-4232-a3ec-a01f1363e764 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9779ffdc7ea45f7b43e2915f467d6af'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:06.963087 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-b02df601-a348-4232-a3ec-a01f1363e764 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9779ffdc7ea45f7b43e2915f467d6af'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:06.963801 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-b02df601-a348-4232-a3ec-a01f1363e764 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9779ffdc7ea45f7b43e2915f467d6af'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:06.976496 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1398/2804] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:38:06 2022] DELETE /identity/v3/users/e9779ffdc7ea45f7b43e2915f467d6af => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:38:07.268492 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6ad137-c618-4097-ac6d-06d7d36a9ab2 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:38:07.269981 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4a6ad137-c618-4097-ac6d-06d7d36a9ab2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:38:07.283528 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6ad137-c618-4097-ac6d-06d7d36a9ab2 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:38:07.344108 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e329d2fd-7d1c-4f81-bfbb-0d79154457e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:38:07.344788 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329d2fd-7d1c-4f81-bfbb-0d79154457e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:07.345061 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329d2fd-7d1c-4f81-bfbb-0d79154457e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:07.345330 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329d2fd-7d1c-4f81-bfbb-0d79154457e0 admin admin] PATH_INFO: `/v3/projects/530ce79d58484c4c98712e932643ddb0` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:07.346508 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e329d2fd-7d1c-4f81-bfbb-0d79154457e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=530ce79d58484c4c98712e932643ddb0)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:38:07.348443 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e329d2fd-7d1c-4f81-bfbb-0d79154457e0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:38:07.380409 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:38:07.398115 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e329d2fd-7d1c-4f81-bfbb-0d79154457e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '530ce79d58484c4c98712e932643ddb0', 'request_id': 'req-e329d2fd-7d1c-4f81-bfbb-0d79154457e0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:07.407449 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e329d2fd-7d1c-4f81-bfbb-0d79154457e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '530ce79d58484c4c98712e932643ddb0', 'request_id': 'req-e329d2fd-7d1c-4f81-bfbb-0d79154457e0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:07.413136 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e329d2fd-7d1c-4f81-bfbb-0d79154457e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '530ce79d58484c4c98712e932643ddb0', 'request_id': 'req-e329d2fd-7d1c-4f81-bfbb-0d79154457e0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:07.424805 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e329d2fd-7d1c-4f81-bfbb-0d79154457e0 admin admin] The token cache is being invalidate because project 530ce79d58484c4c98712e932643ddb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:38:07.425097 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e329d2fd-7d1c-4f81-bfbb-0d79154457e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:07.425814 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e329d2fd-7d1c-4f81-bfbb-0d79154457e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:07.427474 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1407/2805] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:38:07 2022] DELETE /identity/v3/projects/530ce79d58484c4c98712e932643ddb0 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:38:07.432370 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a050a-a452-49bf-8fe7-1ffab098b850 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:07.432884 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a050a-a452-49bf-8fe7-1ffab098b850 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:07.432884 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a050a-a452-49bf-8fe7-1ffab098b850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:07.453491 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-db6a050a-a452-49bf-8fe7-1ffab098b850 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:38:07.454774 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-db6a050a-a452-49bf-8fe7-1ffab098b850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:38:07.516149 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1399/2806] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:38:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 10:38:07.524317 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf70532-2580-4436-a4ff-b1b6e0a2393c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:07.524676 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf70532-2580-4436-a4ff-b1b6e0a2393c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:07.524985 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf70532-2580-4436-a4ff-b1b6e0a2393c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:07.546576 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ebf70532-2580-4436-a4ff-b1b6e0a2393c None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:38:07.547901 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ebf70532-2580-4436-a4ff-b1b6e0a2393c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:38:07.561445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1408/2807] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:38:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:38:07.566352 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6a050a-a452-49bf-8fe7-1ffab098b850 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:38:07.576795 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6a050a-a452-49bf-8fe7-1ffab098b850 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:38:07.582286 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805952a2-7f16-4d96-aef6-c867cdfab104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:38:07.582919 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805952a2-7f16-4d96-aef6-c867cdfab104 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:07.583213 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805952a2-7f16-4d96-aef6-c867cdfab104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:07.583485 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805952a2-7f16-4d96-aef6-c867cdfab104 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:07.584177 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805952a2-7f16-4d96-aef6-c867cdfab104 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:38:07.584480 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805952a2-7f16-4d96-aef6-c867cdfab104 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:38:07.586093 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805952a2-7f16-4d96-aef6-c867cdfab104 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:38:07.598242 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1400/2808] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:38:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:38:07.602682 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf70532-2580-4436-a4ff-b1b6e0a2393c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:38:07.612227 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf70532-2580-4436-a4ff-b1b6e0a2393c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:38:07.618375 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846ef6d4-225f-4580-ad46-f3120ee97008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:38:07.619001 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ef6d4-225f-4580-ad46-f3120ee97008 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:07.619281 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ef6d4-225f-4580-ad46-f3120ee97008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:07.619552 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ef6d4-225f-4580-ad46-f3120ee97008 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:07.620213 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846ef6d4-225f-4580-ad46-f3120ee97008 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:38:07.622134 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846ef6d4-225f-4580-ad46-f3120ee97008 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:38:07.654018 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1409/2809] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:38:07 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:38:07.658477 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805952a2-7f16-4d96-aef6-c867cdfab104 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:38:07.669512 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805952a2-7f16-4d96-aef6-c867cdfab104 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:38:07.701457 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:38:07.740496 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59421c9d-266c-432b-ba1f-cac0f94b5aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:38:07.741137 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59421c9d-266c-432b-ba1f-cac0f94b5aaf admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:07.741414 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59421c9d-266c-432b-ba1f-cac0f94b5aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:07.741681 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59421c9d-266c-432b-ba1f-cac0f94b5aaf admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:07.742349 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59421c9d-266c-432b-ba1f-cac0f94b5aaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:38:07.744292 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59421c9d-266c-432b-ba1f-cac0f94b5aaf admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:38:07.767773 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1401/2810] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:38:07 2022] POST /identity/v3/users => generated 287 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:38:07.772145 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846ef6d4-225f-4580-ad46-f3120ee97008 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:38:07.782548 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846ef6d4-225f-4580-ad46-f3120ee97008 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:38:07.787937 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965647ee-e9fe-4ace-b4f8-a23dec17a146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:38:07.788586 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965647ee-e9fe-4ace-b4f8-a23dec17a146 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:07.788892 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965647ee-e9fe-4ace-b4f8-a23dec17a146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:07.789163 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965647ee-e9fe-4ace-b4f8-a23dec17a146 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:07.789857 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965647ee-e9fe-4ace-b4f8-a23dec17a146 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:38:07.791552 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965647ee-e9fe-4ace-b4f8-a23dec17a146 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:38:07.801970 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1410/2811] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:38:07.805922 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59421c9d-266c-432b-ba1f-cac0f94b5aaf admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:38:07.816625 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59421c9d-266c-432b-ba1f-cac0f94b5aaf admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:38:07.821772 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8b3dcc-f99d-44ba-bb59-2bb9b21c51c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:38:07.822380 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b3dcc-f99d-44ba-bb59-2bb9b21c51c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:07.822662 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b3dcc-f99d-44ba-bb59-2bb9b21c51c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:07.822960 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b3dcc-f99d-44ba-bb59-2bb9b21c51c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:07.823656 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8b3dcc-f99d-44ba-bb59-2bb9b21c51c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:38:07.825254 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8b3dcc-f99d-44ba-bb59-2bb9b21c51c3 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:38:07.835745 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1402/2812] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:38:07.840234 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965647ee-e9fe-4ace-b4f8-a23dec17a146 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:38:07.851813 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965647ee-e9fe-4ace-b4f8-a23dec17a146 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:38:07.857031 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695a310-fec3-469c-8178-4c2676a3d5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:38:07.857717 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695a310-fec3-469c-8178-4c2676a3d5a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:07.857997 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695a310-fec3-469c-8178-4c2676a3d5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:07.858255 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695a310-fec3-469c-8178-4c2676a3d5a4 admin admin] PATH_INFO: `/v3/projects/5fbcf2ab09764a128309ab68d248848b/users/f18929159f6d4ae6bf519d48692aa5f3/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:07.875432 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c695a310-fec3-469c-8178-4c2676a3d5a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fbcf2ab09764a128309ab68d248848b, user_id=f18929159f6d4ae6bf519d48692aa5f3, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:38:07.878135 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c695a310-fec3-469c-8178-4c2676a3d5a4 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:38:07.895087 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1411/2813] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:38:07 2022] PUT /identity/v3/projects/5fbcf2ab09764a128309ab68d248848b/users/f18929159f6d4ae6bf519d48692aa5f3/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:38:07.975299 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2c1cb-4d51-49e2-9582-84e44460a5e6 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:07.975581 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2c1cb-4d51-49e2-9582-84e44460a5e6 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:07.975837 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2c1cb-4d51-49e2-9582-84e44460a5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:08.005444 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-baa2c1cb-4d51-49e2-9582-84e44460a5e6 None None] MFA Rules not processed for user `f18929159f6d4ae6bf519d48692aa5f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:38:08.006573 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-baa2c1cb-4d51-49e2-9582-84e44460a5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:38:08.077178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1403/2814] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:38:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:38:08.081919 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695a310-fec3-469c-8178-4c2676a3d5a4 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:38:08.148604 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695a310-fec3-469c-8178-4c2676a3d5a4 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:38:08.154324 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711008d4-b303-47c1-a04a-4d59f8009a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:38:08.154985 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711008d4-b303-47c1-a04a-4d59f8009a32 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:08.155265 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711008d4-b303-47c1-a04a-4d59f8009a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:08.155534 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711008d4-b303-47c1-a04a-4d59f8009a32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:08.156590 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711008d4-b303-47c1-a04a-4d59f8009a32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:38:08.163834 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711008d4-b303-47c1-a04a-4d59f8009a32 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:38:08.201952 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1412/2815] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:38:08 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:38:08.206252 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa2c1cb-4d51-49e2-9582-84e44460a5e6 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:38:08.217310 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa2c1cb-4d51-49e2-9582-84e44460a5e6 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:38:08.276929 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15720e11-b1b2-4ac7-b0a0-8d3978756598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:38:08.277553 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15720e11-b1b2-4ac7-b0a0-8d3978756598 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:08.277827 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15720e11-b1b2-4ac7-b0a0-8d3978756598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:08.278095 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15720e11-b1b2-4ac7-b0a0-8d3978756598 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:08.278757 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15720e11-b1b2-4ac7-b0a0-8d3978756598 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:38:08.280681 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15720e11-b1b2-4ac7-b0a0-8d3978756598 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:38:08.304397 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1404/2816] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:38:08 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:38:08.308398 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711008d4-b303-47c1-a04a-4d59f8009a32 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:38:08.319964 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711008d4-b303-47c1-a04a-4d59f8009a32 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:38:08.324784 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ff653-0223-478c-be9e-ee8ffe709e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:38:08.325412 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ff653-0223-478c-be9e-ee8ffe709e46 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:08.325688 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ff653-0223-478c-be9e-ee8ffe709e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:08.325958 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ff653-0223-478c-be9e-ee8ffe709e46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:08.326675 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ff653-0223-478c-be9e-ee8ffe709e46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:38:08.328416 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ff653-0223-478c-be9e-ee8ffe709e46 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:38:08.339272 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1413/2817] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:38:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:38:08.343690 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15720e11-b1b2-4ac7-b0a0-8d3978756598 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:38:08.354271 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15720e11-b1b2-4ac7-b0a0-8d3978756598 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:38:08.359020 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a02f5d7-69c9-4b65-b99a-8d2109b7e427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:38:08.359827 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02f5d7-69c9-4b65-b99a-8d2109b7e427 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:08.360027 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02f5d7-69c9-4b65-b99a-8d2109b7e427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:08.360315 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02f5d7-69c9-4b65-b99a-8d2109b7e427 admin admin] PATH_INFO: `/v3/projects/37510a3e4f644b25b8d8b85b96463179/users/47a9a51748dc45048bf1de622815cb1b/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:08.375346 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a02f5d7-69c9-4b65-b99a-8d2109b7e427 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37510a3e4f644b25b8d8b85b96463179, user_id=47a9a51748dc45048bf1de622815cb1b, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:38:08.378144 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a02f5d7-69c9-4b65-b99a-8d2109b7e427 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:38:08.392873 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1405/2818] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:38:08 2022] PUT /identity/v3/projects/37510a3e4f644b25b8d8b85b96463179/users/47a9a51748dc45048bf1de622815cb1b/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:38:08.398248 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ff653-0223-478c-be9e-ee8ffe709e46 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:38:08.408679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ff653-0223-478c-be9e-ee8ffe709e46 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:38:08.428656 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:38:08.469898 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dec3727-aa9a-4313-a179-bc71c2026f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:38:08.470512 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec3727-aa9a-4313-a179-bc71c2026f14 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:08.470788 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec3727-aa9a-4313-a179-bc71c2026f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:08.471089 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec3727-aa9a-4313-a179-bc71c2026f14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:08.471787 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dec3727-aa9a-4313-a179-bc71c2026f14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:38:08.473384 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dec3727-aa9a-4313-a179-bc71c2026f14 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:38:08.483994 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1414/2819] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:38:08 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:38:08.488221 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a02f5d7-69c9-4b65-b99a-8d2109b7e427 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:38:08.493890 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:38:08.499526 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a02f5d7-69c9-4b65-b99a-8d2109b7e427 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:38:08.505753 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e825ecc-a847-4797-a76c-40b70a9fad74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:38:08.506527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e825ecc-a847-4797-a76c-40b70a9fad74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:08.506938 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e825ecc-a847-4797-a76c-40b70a9fad74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:08.507271 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e825ecc-a847-4797-a76c-40b70a9fad74 admin admin] PATH_INFO: `/v3/projects/37510a3e4f644b25b8d8b85b96463179/users/47a9a51748dc45048bf1de622815cb1b/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:08.509317 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e825ecc-a847-4797-a76c-40b70a9fad74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37510a3e4f644b25b8d8b85b96463179, user_id=47a9a51748dc45048bf1de622815cb1b, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:38:08.512384 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e825ecc-a847-4797-a76c-40b70a9fad74 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:38:08.523684 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1406/2820] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:38:08 2022] PUT /identity/v3/projects/37510a3e4f644b25b8d8b85b96463179/users/47a9a51748dc45048bf1de622815cb1b/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:38:08.534832 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628c4cd-30c4-440a-a125-d71b11a3cac6 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:08.535155 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628c4cd-30c4-440a-a125-d71b11a3cac6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:08.535437 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628c4cd-30c4-440a-a125-d71b11a3cac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:08.566012 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c628c4cd-30c4-440a-a125-d71b11a3cac6 None None] MFA Rules not processed for user `47a9a51748dc45048bf1de622815cb1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:38:08.567314 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c628c4cd-30c4-440a-a125-d71b11a3cac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:38:08.640860 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1415/2821] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:38:08 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:38:08.652367 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e825ecc-a847-4797-a76c-40b70a9fad74 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:38:08.659626 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8e825ecc-a847-4797-a76c-40b70a9fad74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:38:08.716842 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e825ecc-a847-4797-a76c-40b70a9fad74 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:38:08.721176 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72ed7f9-dcc6-4680-a8ce-793a8326ca81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:38:08.721802 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ed7f9-dcc6-4680-a8ce-793a8326ca81 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:08.722086 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ed7f9-dcc6-4680-a8ce-793a8326ca81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:08.722359 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ed7f9-dcc6-4680-a8ce-793a8326ca81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:08.780472 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72ed7f9-dcc6-4680-a8ce-793a8326ca81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:38:08.782107 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72ed7f9-dcc6-4680-a8ce-793a8326ca81 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:38:08.791354 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1407/2822] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:38:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:38:11.364381 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c628c4cd-30c4-440a-a125-d71b11a3cac6 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:38:11.430952 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c628c4cd-30c4-440a-a125-d71b11a3cac6 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:38:11.436163 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76e8de-8d98-457e-a177-b5a3a393d420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:38:11.436798 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76e8de-8d98-457e-a177-b5a3a393d420 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:11.437074 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76e8de-8d98-457e-a177-b5a3a393d420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:11.437341 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76e8de-8d98-457e-a177-b5a3a393d420 admin admin] PATH_INFO: `/v3/users/f18929159f6d4ae6bf519d48692aa5f3` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:11.438704 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de76e8de-8d98-457e-a177-b5a3a393d420 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f18929159f6d4ae6bf519d48692aa5f3)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:38:11.440780 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de76e8de-8d98-457e-a177-b5a3a393d420 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:38:11.494893 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-de76e8de-8d98-457e-a177-b5a3a393d420 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f18929159f6d4ae6bf519d48692aa5f3'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:11.495630 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-de76e8de-8d98-457e-a177-b5a3a393d420 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f18929159f6d4ae6bf519d48692aa5f3'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:11.496358 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-de76e8de-8d98-457e-a177-b5a3a393d420 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f18929159f6d4ae6bf519d48692aa5f3'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:11.498013 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:38:11.506023 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-de76e8de-8d98-457e-a177-b5a3a393d420 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f18929159f6d4ae6bf519d48692aa5f3'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:11.513629 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-de76e8de-8d98-457e-a177-b5a3a393d420 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f18929159f6d4ae6bf519d48692aa5f3'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:11.532414 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1416/2823] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:38:11 2022] DELETE /identity/v3/users/f18929159f6d4ae6bf519d48692aa5f3 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:38:11.537332 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29c3c5-cb3c-4808-a1b5-3a9ad77b0ddd None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:11.537806 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29c3c5-cb3c-4808-a1b5-3a9ad77b0ddd None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:11.537926 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29c3c5-cb3c-4808-a1b5-3a9ad77b0ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:11.557982 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-dd29c3c5-cb3c-4808-a1b5-3a9ad77b0ddd None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:38:11.559223 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-dd29c3c5-cb3c-4808-a1b5-3a9ad77b0ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:38:11.618349 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1408/2824] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:38:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:38:11.628493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76e8de-8d98-457e-a177-b5a3a393d420 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:38:11.630036 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-de76e8de-8d98-457e-a177-b5a3a393d420 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:38:11.694536 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76e8de-8d98-457e-a177-b5a3a393d420 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:38:11.698848 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56777c9a-8e31-4709-be0f-f1052cbec492 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:38:11.699493 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56777c9a-8e31-4709-be0f-f1052cbec492 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:11.699777 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56777c9a-8e31-4709-be0f-f1052cbec492 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:11.700048 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56777c9a-8e31-4709-be0f-f1052cbec492 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:11.710486 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56777c9a-8e31-4709-be0f-f1052cbec492 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:38:11.711955 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56777c9a-8e31-4709-be0f-f1052cbec492 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:38:11.722091 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1417/2825] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:38:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:38:12.003464 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd29c3c5-cb3c-4808-a1b5-3a9ad77b0ddd None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:38:12.005092 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-dd29c3c5-cb3c-4808-a1b5-3a9ad77b0ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:38:12.017737 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd29c3c5-cb3c-4808-a1b5-3a9ad77b0ddd None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:38:12.022977 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97a2640-f1db-4c40-89eb-8b2fcd9477f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:38:12.023632 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a2640-f1db-4c40-89eb-8b2fcd9477f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:12.023895 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a2640-f1db-4c40-89eb-8b2fcd9477f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:12.024161 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a2640-f1db-4c40-89eb-8b2fcd9477f3 admin admin] PATH_INFO: `/v3/projects/5fbcf2ab09764a128309ab68d248848b` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:12.025411 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97a2640-f1db-4c40-89eb-8b2fcd9477f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fbcf2ab09764a128309ab68d248848b)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:38:12.027460 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97a2640-f1db-4c40-89eb-8b2fcd9477f3 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:38:12.068568 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:38:12.076736 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c97a2640-f1db-4c40-89eb-8b2fcd9477f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fbcf2ab09764a128309ab68d248848b', 'request_id': 'req-c97a2640-f1db-4c40-89eb-8b2fcd9477f3'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:12.082664 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c97a2640-f1db-4c40-89eb-8b2fcd9477f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fbcf2ab09764a128309ab68d248848b', 'request_id': 'req-c97a2640-f1db-4c40-89eb-8b2fcd9477f3'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:12.091824 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c97a2640-f1db-4c40-89eb-8b2fcd9477f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fbcf2ab09764a128309ab68d248848b', 'request_id': 'req-c97a2640-f1db-4c40-89eb-8b2fcd9477f3'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:12.104082 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c97a2640-f1db-4c40-89eb-8b2fcd9477f3 admin admin] The token cache is being invalidate because project 5fbcf2ab09764a128309ab68d248848b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:38:12.104365 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c97a2640-f1db-4c40-89eb-8b2fcd9477f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:12.105010 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c97a2640-f1db-4c40-89eb-8b2fcd9477f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:12.106523 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1409/2826] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:38:12 2022] DELETE /identity/v3/projects/5fbcf2ab09764a128309ab68d248848b => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:38:12.111042 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56777c9a-8e31-4709-be0f-f1052cbec492 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:38:12.121007 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56777c9a-8e31-4709-be0f-f1052cbec492 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:38:12.182890 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a694c9-a6c5-4201-b46c-6b7f99168428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:38:12.183562 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a694c9-a6c5-4201-b46c-6b7f99168428 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:12.183845 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a694c9-a6c5-4201-b46c-6b7f99168428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:12.184117 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a694c9-a6c5-4201-b46c-6b7f99168428 admin admin] PATH_INFO: `/v3/users/47a9a51748dc45048bf1de622815cb1b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:12.185345 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a694c9-a6c5-4201-b46c-6b7f99168428 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47a9a51748dc45048bf1de622815cb1b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:38:12.187331 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a694c9-a6c5-4201-b46c-6b7f99168428 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:38:12.237998 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-05a694c9-a6c5-4201-b46c-6b7f99168428 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47a9a51748dc45048bf1de622815cb1b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:12.238655 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-05a694c9-a6c5-4201-b46c-6b7f99168428 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47a9a51748dc45048bf1de622815cb1b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:12.239285 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-05a694c9-a6c5-4201-b46c-6b7f99168428 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47a9a51748dc45048bf1de622815cb1b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:12.249625 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-05a694c9-a6c5-4201-b46c-6b7f99168428 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47a9a51748dc45048bf1de622815cb1b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:12.256826 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-05a694c9-a6c5-4201-b46c-6b7f99168428 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47a9a51748dc45048bf1de622815cb1b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:12.262417 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:38:12.277327 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1418/2827] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:38:12 2022] DELETE /identity/v3/users/47a9a51748dc45048bf1de622815cb1b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:38:12.556421 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97a2640-f1db-4c40-89eb-8b2fcd9477f3 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:38:12.557901 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c97a2640-f1db-4c40-89eb-8b2fcd9477f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:38:12.573283 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97a2640-f1db-4c40-89eb-8b2fcd9477f3 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:38:12.630059 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9099f1d6-d459-481f-9e74-2efa8cca97f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:38:12.630739 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099f1d6-d459-481f-9e74-2efa8cca97f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:12.631173 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099f1d6-d459-481f-9e74-2efa8cca97f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:12.631482 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099f1d6-d459-481f-9e74-2efa8cca97f3 admin admin] PATH_INFO: `/v3/projects/37510a3e4f644b25b8d8b85b96463179` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:12.632736 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9099f1d6-d459-481f-9e74-2efa8cca97f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37510a3e4f644b25b8d8b85b96463179)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:38:12.635044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9099f1d6-d459-481f-9e74-2efa8cca97f3 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:38:12.684733 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9099f1d6-d459-481f-9e74-2efa8cca97f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37510a3e4f644b25b8d8b85b96463179', 'request_id': 'req-9099f1d6-d459-481f-9e74-2efa8cca97f3'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:12.691217 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9099f1d6-d459-481f-9e74-2efa8cca97f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37510a3e4f644b25b8d8b85b96463179', 'request_id': 'req-9099f1d6-d459-481f-9e74-2efa8cca97f3'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:12.701463 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9099f1d6-d459-481f-9e74-2efa8cca97f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37510a3e4f644b25b8d8b85b96463179', 'request_id': 'req-9099f1d6-d459-481f-9e74-2efa8cca97f3'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:12.713646 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9099f1d6-d459-481f-9e74-2efa8cca97f3 admin admin] The token cache is being invalidate because project 37510a3e4f644b25b8d8b85b96463179 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:38:12.713945 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9099f1d6-d459-481f-9e74-2efa8cca97f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:12.714558 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9099f1d6-d459-481f-9e74-2efa8cca97f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:12.716126 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1410/2828] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:38:12 2022] DELETE /identity/v3/projects/37510a3e4f644b25b8d8b85b96463179 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:38:12.721678 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05356f55-ab92-4b07-a843-eb9e788bf5b7 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:12.722199 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05356f55-ab92-4b07-a843-eb9e788bf5b7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:12.722199 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05356f55-ab92-4b07-a843-eb9e788bf5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:12.742367 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-05356f55-ab92-4b07-a843-eb9e788bf5b7 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:38:12.743594 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-05356f55-ab92-4b07-a843-eb9e788bf5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:38:12.807429 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1419/2829] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:38:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:38:12.818074 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32e0b2-ec45-4a66-b783-52565a55ea7c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:12.818429 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32e0b2-ec45-4a66-b783-52565a55ea7c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:12.818831 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32e0b2-ec45-4a66-b783-52565a55ea7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:12.847654 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-bf32e0b2-ec45-4a66-b783-52565a55ea7c None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:38:12.848870 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-bf32e0b2-ec45-4a66-b783-52565a55ea7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:38:12.858735 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1411/2830] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:38:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:38:12.863632 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05356f55-ab92-4b07-a843-eb9e788bf5b7 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:38:12.874015 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05356f55-ab92-4b07-a843-eb9e788bf5b7 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:38:12.879390 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2384710a-f829-41af-8b02-230646091c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:38:12.880009 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2384710a-f829-41af-8b02-230646091c20 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:12.880281 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2384710a-f829-41af-8b02-230646091c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:12.880532 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2384710a-f829-41af-8b02-230646091c20 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:12.881225 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2384710a-f829-41af-8b02-230646091c20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:38:12.881522 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2384710a-f829-41af-8b02-230646091c20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:38:12.883113 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2384710a-f829-41af-8b02-230646091c20 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:38:12.896175 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1420/2831] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:38:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:38:12.900845 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf32e0b2-ec45-4a66-b783-52565a55ea7c None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:38:12.911565 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf32e0b2-ec45-4a66-b783-52565a55ea7c None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:38:12.917014 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fdb347-d802-4a41-9032-4af165bb7a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:38:12.917625 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fdb347-d802-4a41-9032-4af165bb7a4f admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:12.917905 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fdb347-d802-4a41-9032-4af165bb7a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:12.918178 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fdb347-d802-4a41-9032-4af165bb7a4f admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:12.918847 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fdb347-d802-4a41-9032-4af165bb7a4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:38:12.920775 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fdb347-d802-4a41-9032-4af165bb7a4f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:38:12.950302 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1412/2832] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:38:12 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:38:12.955530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2384710a-f829-41af-8b02-230646091c20 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:38:12.965917 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2384710a-f829-41af-8b02-230646091c20 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:38:13.025785 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa46086d-6833-4726-b0f9-b697ccf65d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:38:13.026434 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46086d-6833-4726-b0f9-b697ccf65d55 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:13.026711 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46086d-6833-4726-b0f9-b697ccf65d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:13.027019 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46086d-6833-4726-b0f9-b697ccf65d55 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:13.027679 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa46086d-6833-4726-b0f9-b697ccf65d55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:38:13.029583 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa46086d-6833-4726-b0f9-b697ccf65d55 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:38:13.051595 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1421/2833] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:38:12 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:38:13.056745 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fdb347-d802-4a41-9032-4af165bb7a4f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:38:13.069950 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fdb347-d802-4a41-9032-4af165bb7a4f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:38:13.074853 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a14c68-cba8-4341-9e53-dc45d13c11a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:38:13.075573 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a14c68-cba8-4341-9e53-dc45d13c11a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:13.075857 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a14c68-cba8-4341-9e53-dc45d13c11a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:13.076137 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a14c68-cba8-4341-9e53-dc45d13c11a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:13.076832 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a14c68-cba8-4341-9e53-dc45d13c11a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:38:13.078443 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a14c68-cba8-4341-9e53-dc45d13c11a3 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:38:13.088670 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1413/2834] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:38:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:38:13.093212 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa46086d-6833-4726-b0f9-b697ccf65d55 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:38:13.109078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa46086d-6833-4726-b0f9-b697ccf65d55 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:38:13.115612 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d5e12-1e33-4e00-8d31-7d77ec310a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:38:13.116223 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d5e12-1e33-4e00-8d31-7d77ec310a04 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:13.116503 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d5e12-1e33-4e00-8d31-7d77ec310a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:13.116776 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d5e12-1e33-4e00-8d31-7d77ec310a04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:13.117463 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8d5e12-1e33-4e00-8d31-7d77ec310a04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:38:13.119109 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8d5e12-1e33-4e00-8d31-7d77ec310a04 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:38:13.131053 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1422/2835] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:38:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:38:13.137568 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a14c68-cba8-4341-9e53-dc45d13c11a3 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:38:13.149742 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a14c68-cba8-4341-9e53-dc45d13c11a3 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:38:13.155682 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a9e379-240a-4e91-94fd-fe83db2afb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:38:13.156364 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9e379-240a-4e91-94fd-fe83db2afb78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:13.156638 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9e379-240a-4e91-94fd-fe83db2afb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:13.156892 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9e379-240a-4e91-94fd-fe83db2afb78 admin admin] PATH_INFO: `/v3/projects/3977bfbc156b44c7a7ba0c273708b2b1/users/20886161e1f34eedbea38b1ac7efed50/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:13.174336 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a9e379-240a-4e91-94fd-fe83db2afb78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3977bfbc156b44c7a7ba0c273708b2b1, user_id=20886161e1f34eedbea38b1ac7efed50, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:38:13.177003 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a9e379-240a-4e91-94fd-fe83db2afb78 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:38:13.179207 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:38:13.198284 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1414/2836] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:38:13 2022] PUT /identity/v3/projects/3977bfbc156b44c7a7ba0c273708b2b1/users/20886161e1f34eedbea38b1ac7efed50/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:38:13.210146 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74956ed3-3ed9-4844-9999-abac96cbb3f4 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:13.210432 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74956ed3-3ed9-4844-9999-abac96cbb3f4 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:13.210713 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74956ed3-3ed9-4844-9999-abac96cbb3f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:13.239732 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-74956ed3-3ed9-4844-9999-abac96cbb3f4 None None] MFA Rules not processed for user `20886161e1f34eedbea38b1ac7efed50`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:38:13.240939 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-74956ed3-3ed9-4844-9999-abac96cbb3f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:38:13.293767 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:38:13.319351 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1423/2837] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:38:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:38:13.331042 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a9e379-240a-4e91-94fd-fe83db2afb78 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:38:13.332954 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-43a9e379-240a-4e91-94fd-fe83db2afb78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:38:13.390753 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a9e379-240a-4e91-94fd-fe83db2afb78 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:38:13.394777 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b294c2ea-0560-45f4-bc38-9dc6962e0d0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:38:13.395427 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294c2ea-0560-45f4-bc38-9dc6962e0d0a service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:13.395706 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294c2ea-0560-45f4-bc38-9dc6962e0d0a service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:13.395975 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294c2ea-0560-45f4-bc38-9dc6962e0d0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:13.404655 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b294c2ea-0560-45f4-bc38-9dc6962e0d0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:38:13.406080 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b294c2ea-0560-45f4-bc38-9dc6962e0d0a service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:38:13.414822 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1415/2838] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:38:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:38:14.810890 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8505fae-6a36-41d8-b94b-27b6e767e245 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:14.811484 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8505fae-6a36-41d8-b94b-27b6e767e245 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:14.811484 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8505fae-6a36-41d8-b94b-27b6e767e245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:14.831623 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b8505fae-6a36-41d8-b94b-27b6e767e245 None None] MFA Rules not processed for user `20886161e1f34eedbea38b1ac7efed50`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:38:14.832829 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b8505fae-6a36-41d8-b94b-27b6e767e245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:38:14.841380 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1424/2839] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:38:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:38:15.340576 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b294c2ea-0560-45f4-bc38-9dc6962e0d0a service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:38:15.342289 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b294c2ea-0560-45f4-bc38-9dc6962e0d0a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:38:15.419111 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b294c2ea-0560-45f4-bc38-9dc6962e0d0a service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:38:15.424599 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2106f2c-2c86-4bc6-9737-e5ef16d8da66 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8743f5bb0be54d57a957617636b0b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:38:15.425291 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2106f2c-2c86-4bc6-9737-e5ef16d8da66 service placement] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:15.425696 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2106f2c-2c86-4bc6-9737-e5ef16d8da66 service placement] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:15.425965 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2106f2c-2c86-4bc6-9737-e5ef16d8da66 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:15.427832 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b2106f2c-2c86-4bc6-9737-e5ef16d8da66 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:38:15.440436 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2106f2c-2c86-4bc6-9737-e5ef16d8da66 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:38:15.442064 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2106f2c-2c86-4bc6-9737-e5ef16d8da66 service placement] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:38:15.452526 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1416/2840] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:38:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:38:24.825736 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=11 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:38:25.448004 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:INSERT=1,keystone:SELECT=60 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:38:41.712765 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbac91f-fa3b-4d59-8ce7-514aed8100c3 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:41.713230 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbac91f-fa3b-4d59-8ce7-514aed8100c3 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:41.713325 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbac91f-fa3b-4d59-8ce7-514aed8100c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:41.766230 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-4bbac91f-fa3b-4d59-8ce7-514aed8100c3 None None] MFA Rules not processed for user `52fc3ba90dc045c2a5b7bb730da96acc`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:38:41.767465 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4bbac91f-fa3b-4d59-8ce7-514aed8100c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:38:41.844883 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1425/2841] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 5 10:38:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:38:41.897470 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2106f2c-2c86-4bc6-9737-e5ef16d8da66 service placement] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:38:41.899522 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b2106f2c-2c86-4bc6-9737-e5ef16d8da66 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:38:41.997615 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2106f2c-2c86-4bc6-9737-e5ef16d8da66 service placement] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:38:42.005076 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a2530-f72c-4c5e-8bb0-04f1c657aaeb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c80cf0e429f04f229aea1aa9293a93b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:38:42.005951 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a2530-f72c-4c5e-8bb0-04f1c657aaeb service swift] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:42.006368 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a2530-f72c-4c5e-8bb0-04f1c657aaeb service swift] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:42.006705 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a2530-f72c-4c5e-8bb0-04f1c657aaeb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:42.016157 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a2530-f72c-4c5e-8bb0-04f1c657aaeb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:38:42.017727 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a2530-f72c-4c5e-8bb0-04f1c657aaeb service swift] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:38:42.026269 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1417/2842] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 5 10:38:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 10:38:42.564489 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f16eec-7dcb-4226-bc30-b7c80a4a062b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:42.564779 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f16eec-7dcb-4226-bc30-b7c80a4a062b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:42.565065 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f16eec-7dcb-4226-bc30-b7c80a4a062b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:42.585857 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-91f16eec-7dcb-4226-bc30-b7c80a4a062b None None] MFA Rules not processed for user `52fc3ba90dc045c2a5b7bb730da96acc`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:38:42.587101 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-91f16eec-7dcb-4226-bc30-b7c80a4a062b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:38:42.594964 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1426/2843] 10.222.0.40 () {58 vars in 967 bytes} [Fri Aug 5 10:38:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:38:42.608219 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a2530-f72c-4c5e-8bb0-04f1c657aaeb service swift] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:38:42.618894 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a2530-f72c-4c5e-8bb0-04f1c657aaeb service swift] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:38:42.624049 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b53e2-9790-4557-b8d0-1175935a9ae1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c80cf0e429f04f229aea1aa9293a93b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:38:42.624597 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b53e2-9790-4557-b8d0-1175935a9ae1 service swift] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:42.624881 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b53e2-9790-4557-b8d0-1175935a9ae1 service swift] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:42.625155 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b53e2-9790-4557-b8d0-1175935a9ae1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:42.633805 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2b53e2-9790-4557-b8d0-1175935a9ae1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:38:42.635402 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2b53e2-9790-4557-b8d0-1175935a9ae1 service swift] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:38:42.643643 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1418/2844] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 5 10:38:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 10:38:42.882327 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7bf85-28dc-43ec-8116-66a5fe73b800 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:42.882892 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7bf85-28dc-43ec-8116-66a5fe73b800 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:42.883124 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7bf85-28dc-43ec-8116-66a5fe73b800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:42.903412 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d7a7bf85-28dc-43ec-8116-66a5fe73b800 None None] MFA Rules not processed for user `52fc3ba90dc045c2a5b7bb730da96acc`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:38:42.904723 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d7a7bf85-28dc-43ec-8116-66a5fe73b800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:38:42.911896 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1427/2845] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 5 10:38:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:38:42.923929 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b53e2-9790-4557-b8d0-1175935a9ae1 service swift] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:38:42.935084 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b53e2-9790-4557-b8d0-1175935a9ae1 service swift] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:38:42.941266 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e3a387-34c9-4517-9d16-7af68961fd29 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c80cf0e429f04f229aea1aa9293a93b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:38:42.941878 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3a387-34c9-4517-9d16-7af68961fd29 service swift] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:42.942164 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3a387-34c9-4517-9d16-7af68961fd29 service swift] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:42.942433 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3a387-34c9-4517-9d16-7af68961fd29 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:42.951037 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e3a387-34c9-4517-9d16-7af68961fd29 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:38:42.952612 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e3a387-34c9-4517-9d16-7af68961fd29 service swift] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:38:42.960921 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1419/2846] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 5 10:38:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 10:38:43.301411 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120934ce-c64d-4dfe-88a6-055d3d803558 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:43.301764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120934ce-c64d-4dfe-88a6-055d3d803558 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:43.302110 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120934ce-c64d-4dfe-88a6-055d3d803558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:43.321880 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-120934ce-c64d-4dfe-88a6-055d3d803558 None None] MFA Rules not processed for user `52fc3ba90dc045c2a5b7bb730da96acc`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:38:43.323112 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-120934ce-c64d-4dfe-88a6-055d3d803558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:38:43.330581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1428/2847] 10.222.0.40 () {58 vars in 967 bytes} [Fri Aug 5 10:38:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:38:43.343068 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e3a387-34c9-4517-9d16-7af68961fd29 service swift] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:38:43.355032 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e3a387-34c9-4517-9d16-7af68961fd29 service swift] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:38:43.360309 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cadbf8-d926-4c59-b510-cdcead579e85 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c80cf0e429f04f229aea1aa9293a93b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:38:43.360934 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cadbf8-d926-4c59-b510-cdcead579e85 service swift] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:43.361215 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cadbf8-d926-4c59-b510-cdcead579e85 service swift] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:43.361488 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cadbf8-d926-4c59-b510-cdcead579e85 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:43.369806 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cadbf8-d926-4c59-b510-cdcead579e85 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:38:43.371482 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cadbf8-d926-4c59-b510-cdcead579e85 service swift] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:38:43.379739 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1420/2848] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 5 10:38:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 10:38:47.841678 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120934ce-c64d-4dfe-88a6-055d3d803558 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:38:47.913863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120934ce-c64d-4dfe-88a6-055d3d803558 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:38:47.919206 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddde966-4e51-4af9-8046-3967bfd09cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:38:47.919837 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddde966-4e51-4af9-8046-3967bfd09cd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:47.920115 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddde966-4e51-4af9-8046-3967bfd09cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:47.920394 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddde966-4e51-4af9-8046-3967bfd09cd3 admin admin] PATH_INFO: `/v3/users/20886161e1f34eedbea38b1ac7efed50` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:47.922006 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddde966-4e51-4af9-8046-3967bfd09cd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20886161e1f34eedbea38b1ac7efed50)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:38:47.923972 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddde966-4e51-4af9-8046-3967bfd09cd3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:38:47.975863 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0ddde966-4e51-4af9-8046-3967bfd09cd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20886161e1f34eedbea38b1ac7efed50'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:47.976631 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0ddde966-4e51-4af9-8046-3967bfd09cd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20886161e1f34eedbea38b1ac7efed50'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:47.977350 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0ddde966-4e51-4af9-8046-3967bfd09cd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20886161e1f34eedbea38b1ac7efed50'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:47.986923 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0ddde966-4e51-4af9-8046-3967bfd09cd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20886161e1f34eedbea38b1ac7efed50'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:47.988427 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:38:47.995313 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0ddde966-4e51-4af9-8046-3967bfd09cd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20886161e1f34eedbea38b1ac7efed50'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:48.015516 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1429/2849] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:38:47 2022] DELETE /identity/v3/users/20886161e1f34eedbea38b1ac7efed50 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:38:48.020238 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b3a69-cfbc-4814-97e6-cf0bc660141d None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:48.020751 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b3a69-cfbc-4814-97e6-cf0bc660141d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:48.020751 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b3a69-cfbc-4814-97e6-cf0bc660141d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:48.043905 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-4c7b3a69-cfbc-4814-97e6-cf0bc660141d None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:38:48.045121 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4c7b3a69-cfbc-4814-97e6-cf0bc660141d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:38:48.118417 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1421/2850] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:38:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:38:48.129703 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddde966-4e51-4af9-8046-3967bfd09cd3 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:38:48.131427 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0ddde966-4e51-4af9-8046-3967bfd09cd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:38:48.197173 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddde966-4e51-4af9-8046-3967bfd09cd3 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:38:48.201723 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa64ab0-e214-4f7e-81e4-e65338e0d85f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:38:48.202371 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa64ab0-e214-4f7e-81e4-e65338e0d85f service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:48.202655 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa64ab0-e214-4f7e-81e4-e65338e0d85f service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:48.202934 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa64ab0-e214-4f7e-81e4-e65338e0d85f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:48.212687 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa64ab0-e214-4f7e-81e4-e65338e0d85f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:38:48.214136 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa64ab0-e214-4f7e-81e4-e65338e0d85f service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:38:48.224057 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1430/2851] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:38:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:38:48.409149 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b3a69-cfbc-4814-97e6-cf0bc660141d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:38:48.412170 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4c7b3a69-cfbc-4814-97e6-cf0bc660141d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:38:48.429523 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b3a69-cfbc-4814-97e6-cf0bc660141d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:38:48.438849 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4d14ab-2df3-43c4-84c5-add7808e50e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:38:48.439821 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d14ab-2df3-43c4-84c5-add7808e50e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:48.440309 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d14ab-2df3-43c4-84c5-add7808e50e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:48.440710 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d14ab-2df3-43c4-84c5-add7808e50e0 admin admin] PATH_INFO: `/v3/projects/3977bfbc156b44c7a7ba0c273708b2b1` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:48.441714 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4d14ab-2df3-43c4-84c5-add7808e50e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3977bfbc156b44c7a7ba0c273708b2b1)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:38:48.443945 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4d14ab-2df3-43c4-84c5-add7808e50e0 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:38:48.476909 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:38:48.493825 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5e4d14ab-2df3-43c4-84c5-add7808e50e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3977bfbc156b44c7a7ba0c273708b2b1', 'request_id': 'req-5e4d14ab-2df3-43c4-84c5-add7808e50e0'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:48.499847 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5e4d14ab-2df3-43c4-84c5-add7808e50e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3977bfbc156b44c7a7ba0c273708b2b1', 'request_id': 'req-5e4d14ab-2df3-43c4-84c5-add7808e50e0'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:48.508819 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5e4d14ab-2df3-43c4-84c5-add7808e50e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3977bfbc156b44c7a7ba0c273708b2b1', 'request_id': 'req-5e4d14ab-2df3-43c4-84c5-add7808e50e0'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:48.520592 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5e4d14ab-2df3-43c4-84c5-add7808e50e0 admin admin] The token cache is being invalidate because project 3977bfbc156b44c7a7ba0c273708b2b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:38:48.520879 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5e4d14ab-2df3-43c4-84c5-add7808e50e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:48.521628 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5e4d14ab-2df3-43c4-84c5-add7808e50e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:38:48.523373 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1422/2852] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:38:48 2022] DELETE /identity/v3/projects/3977bfbc156b44c7a7ba0c273708b2b1 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:38:52.600506 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a386d-d633-4cf6-9c72-8497b6879000 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:52.601059 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a386d-d633-4cf6-9c72-8497b6879000 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:52.601059 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a386d-d633-4cf6-9c72-8497b6879000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:52.621183 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-675a386d-d633-4cf6-9c72-8497b6879000 None None] MFA Rules not processed for user `85904ab7159f486ea9d5075b285148f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:38:52.622381 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-675a386d-d633-4cf6-9c72-8497b6879000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:38:52.680975 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1431/2853] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:38:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:38:52.927738 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4d14ab-2df3-43c4-84c5-add7808e50e0 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:38:52.929382 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5e4d14ab-2df3-43c4-84c5-add7808e50e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:38:52.987963 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4d14ab-2df3-43c4-84c5-add7808e50e0 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:38:52.991879 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf88531-879e-49a3-9264-356c1c3b5017 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:38:52.992486 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf88531-879e-49a3-9264-356c1c3b5017 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:38:52.992767 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf88531-879e-49a3-9264-356c1c3b5017 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:38:52.993014 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf88531-879e-49a3-9264-356c1c3b5017 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:38:52.994810 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3bf88531-879e-49a3-9264-356c1c3b5017 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:38:53.004408 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf88531-879e-49a3-9264-356c1c3b5017 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:38:53.005820 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf88531-879e-49a3-9264-356c1c3b5017 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:38:53.014070 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1423/2854] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:38:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:39:02.673701 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=62,keystone:DELETE=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:39:03.010821 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:DELETE=6,keystone:SELECT=40,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:39:57.431077 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a386d-d633-4cf6-9c72-8497b6879000 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:39:57.432615 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-675a386d-d633-4cf6-9c72-8497b6879000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:39:57.497648 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf88531-879e-49a3-9264-356c1c3b5017 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:39:57.499393 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3bf88531-879e-49a3-9264-356c1c3b5017 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:39:57.508581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a386d-d633-4cf6-9c72-8497b6879000 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:39:57.514177 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb99e5-d10f-4be4-9851-b67c26382d9a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8743f5bb0be54d57a957617636b0b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:39:57.514792 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb99e5-d10f-4be4-9851-b67c26382d9a service placement] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:39:57.515114 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb99e5-d10f-4be4-9851-b67c26382d9a service placement] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:39:57.515382 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb99e5-d10f-4be4-9851-b67c26382d9a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:39:57.517262 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-41eb99e5-d10f-4be4-9851-b67c26382d9a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:39:57.531113 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eb99e5-d10f-4be4-9851-b67c26382d9a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:39:57.532848 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eb99e5-d10f-4be4-9851-b67c26382d9a service placement] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:39:57.543499 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1432/2855] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:39:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:39:57.578645 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf88531-879e-49a3-9264-356c1c3b5017 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:39:57.583622 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60bcba7-a0f9-4f42-934d-f71e4b6d639b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:39:57.584271 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60bcba7-a0f9-4f42-934d-f71e4b6d639b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:39:57.584554 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60bcba7-a0f9-4f42-934d-f71e4b6d639b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:39:57.584830 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60bcba7-a0f9-4f42-934d-f71e4b6d639b admin admin] PATH_INFO: `/v3/users/85904ab7159f486ea9d5075b285148f7` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:39:57.586071 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60bcba7-a0f9-4f42-934d-f71e4b6d639b admin admin] RBAC: Authorizing `identity:delete_user(user_id=85904ab7159f486ea9d5075b285148f7)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:39:57.588044 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60bcba7-a0f9-4f42-934d-f71e4b6d639b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:39:57.639002 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f60bcba7-a0f9-4f42-934d-f71e4b6d639b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85904ab7159f486ea9d5075b285148f7'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:39:57.647975 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f60bcba7-a0f9-4f42-934d-f71e4b6d639b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85904ab7159f486ea9d5075b285148f7'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:39:57.648622 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f60bcba7-a0f9-4f42-934d-f71e4b6d639b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85904ab7159f486ea9d5075b285148f7'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:39:57.660546 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f60bcba7-a0f9-4f42-934d-f71e4b6d639b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85904ab7159f486ea9d5075b285148f7'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:39:57.661177 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f60bcba7-a0f9-4f42-934d-f71e4b6d639b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85904ab7159f486ea9d5075b285148f7'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:39:57.673676 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1424/2856] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:39:57 2022] DELETE /identity/v3/users/85904ab7159f486ea9d5075b285148f7 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:39:57.891619 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb99e5-d10f-4be4-9851-b67c26382d9a service placement] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:39:57.905484 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb99e5-d10f-4be4-9851-b67c26382d9a service placement] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:39:57.988565 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef19015-2de0-43e9-bddf-412ed4269d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:39:57.989223 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef19015-2de0-43e9-bddf-412ed4269d90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:39:57.989503 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef19015-2de0-43e9-bddf-412ed4269d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:39:57.989774 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef19015-2de0-43e9-bddf-412ed4269d90 admin admin] PATH_INFO: `/v3/projects/2bbcb32efae644d8b8ab1c92fb168a7b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:39:57.991167 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef19015-2de0-43e9-bddf-412ed4269d90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bbcb32efae644d8b8ab1c92fb168a7b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:39:57.993091 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef19015-2de0-43e9-bddf-412ed4269d90 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:39:58.050773 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5ef19015-2de0-43e9-bddf-412ed4269d90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bbcb32efae644d8b8ab1c92fb168a7b', 'request_id': 'req-5ef19015-2de0-43e9-bddf-412ed4269d90'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:39:58.059581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5ef19015-2de0-43e9-bddf-412ed4269d90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bbcb32efae644d8b8ab1c92fb168a7b', 'request_id': 'req-5ef19015-2de0-43e9-bddf-412ed4269d90'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:39:58.064930 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5ef19015-2de0-43e9-bddf-412ed4269d90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bbcb32efae644d8b8ab1c92fb168a7b', 'request_id': 'req-5ef19015-2de0-43e9-bddf-412ed4269d90'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:39:58.076017 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5ef19015-2de0-43e9-bddf-412ed4269d90 admin admin] The token cache is being invalidate because project 2bbcb32efae644d8b8ab1c92fb168a7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:39:58.076305 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5ef19015-2de0-43e9-bddf-412ed4269d90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:39:58.077064 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5ef19015-2de0-43e9-bddf-412ed4269d90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:39:58.078747 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1433/2857] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:39:57 2022] DELETE /identity/v3/projects/2bbcb32efae644d8b8ab1c92fb168a7b => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:39:58.083955 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a68d4-bbd9-407d-a190-b3bae272a629 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:39:58.084241 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a68d4-bbd9-407d-a190-b3bae272a629 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:39:58.084514 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a68d4-bbd9-407d-a190-b3bae272a629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:39:58.105754 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-8a4a68d4-bbd9-407d-a190-b3bae272a629 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:39:58.106993 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8a4a68d4-bbd9-407d-a190-b3bae272a629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:39:58.171311 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1425/2858] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:39:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:39:58.180429 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4da06f5-d95e-481f-b494-cd232ea364ba None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:39:58.180732 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4da06f5-d95e-481f-b494-cd232ea364ba None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:39:58.181015 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4da06f5-d95e-481f-b494-cd232ea364ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:39:58.201319 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b4da06f5-d95e-481f-b494-cd232ea364ba None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:39:58.202518 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b4da06f5-d95e-481f-b494-cd232ea364ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:39:58.211308 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1434/2859] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:39:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:39:58.215621 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4a68d4-bbd9-407d-a190-b3bae272a629 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:39:58.225768 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4a68d4-bbd9-407d-a190-b3bae272a629 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:39:58.231085 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666437c9-21c5-4ce9-8c15-188e3879270d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:39:58.231692 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666437c9-21c5-4ce9-8c15-188e3879270d admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:39:58.231967 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666437c9-21c5-4ce9-8c15-188e3879270d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:39:58.232231 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666437c9-21c5-4ce9-8c15-188e3879270d admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:39:58.232952 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666437c9-21c5-4ce9-8c15-188e3879270d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:39:58.233252 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666437c9-21c5-4ce9-8c15-188e3879270d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:39:58.234926 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666437c9-21c5-4ce9-8c15-188e3879270d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:39:58.247581 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1426/2860] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:39:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:39:58.252116 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4da06f5-d95e-481f-b494-cd232ea364ba None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:39:58.262217 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4da06f5-d95e-481f-b494-cd232ea364ba None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:39:58.267673 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841c159-0f80-4b14-9046-04fc0ef7a951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:39:58.268279 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841c159-0f80-4b14-9046-04fc0ef7a951 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:39:58.268538 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841c159-0f80-4b14-9046-04fc0ef7a951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:39:58.268824 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841c159-0f80-4b14-9046-04fc0ef7a951 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:39:58.269482 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9841c159-0f80-4b14-9046-04fc0ef7a951 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:39:58.271406 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9841c159-0f80-4b14-9046-04fc0ef7a951 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:39:58.283193 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:39:58.308309 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1435/2861] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:39:58 2022] POST /identity/v3/projects => generated 391 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:39:58.312551 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666437c9-21c5-4ce9-8c15-188e3879270d admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:39:58.323384 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666437c9-21c5-4ce9-8c15-188e3879270d admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:39:58.327333 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:39:58.386628 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8357c33-04ee-40fa-9013-339d2c556fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:39:58.387279 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8357c33-04ee-40fa-9013-339d2c556fda admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:39:58.387560 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8357c33-04ee-40fa-9013-339d2c556fda admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:39:58.387830 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8357c33-04ee-40fa-9013-339d2c556fda admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:39:58.388494 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8357c33-04ee-40fa-9013-339d2c556fda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:39:58.390408 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8357c33-04ee-40fa-9013-339d2c556fda admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:39:58.414380 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1427/2862] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:39:58 2022] POST /identity/v3/users => generated 296 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:39:58.418712 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841c159-0f80-4b14-9046-04fc0ef7a951 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:39:58.429874 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841c159-0f80-4b14-9046-04fc0ef7a951 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:39:58.434592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554410d0-e0f5-4d93-8896-0d46427c5805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:39:58.435241 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554410d0-e0f5-4d93-8896-0d46427c5805 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:39:58.435521 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554410d0-e0f5-4d93-8896-0d46427c5805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:39:58.435795 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554410d0-e0f5-4d93-8896-0d46427c5805 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:39:58.436516 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554410d0-e0f5-4d93-8896-0d46427c5805 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:39:58.438128 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554410d0-e0f5-4d93-8896-0d46427c5805 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:39:58.448688 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1436/2863] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:39:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:39:58.452573 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8357c33-04ee-40fa-9013-339d2c556fda admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:39:58.463114 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8357c33-04ee-40fa-9013-339d2c556fda admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:39:58.468321 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0355459-abf1-4d2c-bef3-0da07671d737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:39:58.468930 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0355459-abf1-4d2c-bef3-0da07671d737 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:39:58.469211 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0355459-abf1-4d2c-bef3-0da07671d737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:39:58.469483 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0355459-abf1-4d2c-bef3-0da07671d737 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:39:58.470732 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0355459-abf1-4d2c-bef3-0da07671d737 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:39:58.472364 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0355459-abf1-4d2c-bef3-0da07671d737 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:39:58.482787 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1428/2864] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:39:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:39:58.487120 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554410d0-e0f5-4d93-8896-0d46427c5805 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:39:58.497916 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554410d0-e0f5-4d93-8896-0d46427c5805 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:39:58.504066 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62a3a4-021e-49a6-9b17-9e32d3f3f2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:39:58.505078 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62a3a4-021e-49a6-9b17-9e32d3f3f2e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:39:58.505350 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62a3a4-021e-49a6-9b17-9e32d3f3f2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:39:58.505621 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62a3a4-021e-49a6-9b17-9e32d3f3f2e1 admin admin] PATH_INFO: `/v3/projects/5cd6b302c8d44b84b29ae34d1e36588d/users/e67108fe77fb44c69629947c6cea4ad1/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:39:58.521808 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62a3a4-021e-49a6-9b17-9e32d3f3f2e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cd6b302c8d44b84b29ae34d1e36588d, user_id=e67108fe77fb44c69629947c6cea4ad1, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:39:58.524516 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62a3a4-021e-49a6-9b17-9e32d3f3f2e1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:39:58.539533 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1437/2865] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:39:58 2022] PUT /identity/v3/projects/5cd6b302c8d44b84b29ae34d1e36588d/users/e67108fe77fb44c69629947c6cea4ad1/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:39:58.549999 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c52d4e-b2ab-4e25-ac47-b5bcb2e29ae7 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:39:58.550488 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c52d4e-b2ab-4e25-ac47-b5bcb2e29ae7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:39:58.550607 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c52d4e-b2ab-4e25-ac47-b5bcb2e29ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:39:58.579701 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a9c52d4e-b2ab-4e25-ac47-b5bcb2e29ae7 None None] MFA Rules not processed for user `e67108fe77fb44c69629947c6cea4ad1`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:39:58.580847 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a9c52d4e-b2ab-4e25-ac47-b5bcb2e29ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:39:58.649271 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1429/2866] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:39:58 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:39:58.660857 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62a3a4-021e-49a6-9b17-9e32d3f3f2e1 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:39:58.662857 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4e62a3a4-021e-49a6-9b17-9e32d3f3f2e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:39:58.721032 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62a3a4-021e-49a6-9b17-9e32d3f3f2e1 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:39:58.724997 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466693e7-4d22-4c87-bd44-530349c84855 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:39:58.725620 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466693e7-4d22-4c87-bd44-530349c84855 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:39:58.725904 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466693e7-4d22-4c87-bd44-530349c84855 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:39:58.726175 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466693e7-4d22-4c87-bd44-530349c84855 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:39:58.734919 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466693e7-4d22-4c87-bd44-530349c84855 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:39:58.736562 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466693e7-4d22-4c87-bd44-530349c84855 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:39:58.745736 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1438/2867] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:39:58 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:40:08.646128 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=63,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:40:08.741843 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=44,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:40:19.327812 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c52d4e-b2ab-4e25-ac47-b5bcb2e29ae7 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:40:19.397766 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c52d4e-b2ab-4e25-ac47-b5bcb2e29ae7 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:40:19.403696 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d6a5b9-6bf0-46dd-984a-b741e866e578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:40:19.404359 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6a5b9-6bf0-46dd-984a-b741e866e578 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:40:19.404638 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6a5b9-6bf0-46dd-984a-b741e866e578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:40:19.404911 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6a5b9-6bf0-46dd-984a-b741e866e578 admin admin] PATH_INFO: `/v3/users/e67108fe77fb44c69629947c6cea4ad1` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:40:19.406142 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d6a5b9-6bf0-46dd-984a-b741e866e578 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e67108fe77fb44c69629947c6cea4ad1)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:40:19.408098 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d6a5b9-6bf0-46dd-984a-b741e866e578 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:40:19.460001 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f8d6a5b9-6bf0-46dd-984a-b741e866e578 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e67108fe77fb44c69629947c6cea4ad1'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:40:19.469208 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f8d6a5b9-6bf0-46dd-984a-b741e866e578 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e67108fe77fb44c69629947c6cea4ad1'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:40:19.469807 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f8d6a5b9-6bf0-46dd-984a-b741e866e578 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e67108fe77fb44c69629947c6cea4ad1'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:40:19.481715 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f8d6a5b9-6bf0-46dd-984a-b741e866e578 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e67108fe77fb44c69629947c6cea4ad1'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:40:19.482381 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f8d6a5b9-6bf0-46dd-984a-b741e866e578 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e67108fe77fb44c69629947c6cea4ad1'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:40:19.494696 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1430/2868] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:40:19 2022] DELETE /identity/v3/users/e67108fe77fb44c69629947c6cea4ad1 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:40:19.499613 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987de93-f39f-4967-8fc8-6086d4dabce0 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:40:19.500112 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987de93-f39f-4967-8fc8-6086d4dabce0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:40:19.500219 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987de93-f39f-4967-8fc8-6086d4dabce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:40:19.521442 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f987de93-f39f-4967-8fc8-6086d4dabce0 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:40:19.522641 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f987de93-f39f-4967-8fc8-6086d4dabce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:40:19.583889 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1439/2869] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:40:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:40:19.593855 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d6a5b9-6bf0-46dd-984a-b741e866e578 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:40:19.595491 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f8d6a5b9-6bf0-46dd-984a-b741e866e578 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:40:19.655915 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d6a5b9-6bf0-46dd-984a-b741e866e578 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:40:19.659809 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2d99eb-05a0-44ae-af82-205f1b2a256e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:40:19.660428 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2d99eb-05a0-44ae-af82-205f1b2a256e service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:40:19.660713 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2d99eb-05a0-44ae-af82-205f1b2a256e service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:40:19.660984 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2d99eb-05a0-44ae-af82-205f1b2a256e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:40:19.670328 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2d99eb-05a0-44ae-af82-205f1b2a256e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:40:19.671789 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2d99eb-05a0-44ae-af82-205f1b2a256e service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:40:19.681249 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1431/2870] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:40:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:40:19.958712 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f987de93-f39f-4967-8fc8-6086d4dabce0 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:40:19.960282 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f987de93-f39f-4967-8fc8-6086d4dabce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:40:19.973029 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f987de93-f39f-4967-8fc8-6086d4dabce0 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:40:19.978803 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f319ff94-d68f-4cc9-a1b1-69a33b47a4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:40:19.979480 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319ff94-d68f-4cc9-a1b1-69a33b47a4ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:40:19.979756 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319ff94-d68f-4cc9-a1b1-69a33b47a4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:40:19.980024 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319ff94-d68f-4cc9-a1b1-69a33b47a4ca admin admin] PATH_INFO: `/v3/projects/5cd6b302c8d44b84b29ae34d1e36588d` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:40:19.981322 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f319ff94-d68f-4cc9-a1b1-69a33b47a4ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cd6b302c8d44b84b29ae34d1e36588d)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:40:19.983259 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f319ff94-d68f-4cc9-a1b1-69a33b47a4ca admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:40:20.034300 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f319ff94-d68f-4cc9-a1b1-69a33b47a4ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cd6b302c8d44b84b29ae34d1e36588d', 'request_id': 'req-f319ff94-d68f-4cc9-a1b1-69a33b47a4ca'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:40:20.043008 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f319ff94-d68f-4cc9-a1b1-69a33b47a4ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cd6b302c8d44b84b29ae34d1e36588d', 'request_id': 'req-f319ff94-d68f-4cc9-a1b1-69a33b47a4ca'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:40:20.048862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f319ff94-d68f-4cc9-a1b1-69a33b47a4ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cd6b302c8d44b84b29ae34d1e36588d', 'request_id': 'req-f319ff94-d68f-4cc9-a1b1-69a33b47a4ca'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:40:20.060527 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f319ff94-d68f-4cc9-a1b1-69a33b47a4ca admin admin] The token cache is being invalidate because project 5cd6b302c8d44b84b29ae34d1e36588d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:40:20.060810 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f319ff94-d68f-4cc9-a1b1-69a33b47a4ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:40:20.061514 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f319ff94-d68f-4cc9-a1b1-69a33b47a4ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:40:20.063082 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1440/2871] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:40:19 2022] DELETE /identity/v3/projects/5cd6b302c8d44b84b29ae34d1e36588d => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:40:20.069793 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d1284-7e86-4941-8abe-d5c5bdf65bd9 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:40:20.070079 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d1284-7e86-4941-8abe-d5c5bdf65bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:40:20.070359 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d1284-7e86-4941-8abe-d5c5bdf65bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:40:20.094590 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-1b2d1284-7e86-4941-8abe-d5c5bdf65bd9 None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:40:20.095852 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1b2d1284-7e86-4941-8abe-d5c5bdf65bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:40:20.114490 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:40:20.160482 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1432/2872] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:40:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:40:20.169655 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc89a5-4465-499f-94a8-42da58f3e36f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:40:20.169992 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc89a5-4465-499f-94a8-42da58f3e36f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:40:20.170363 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc89a5-4465-499f-94a8-42da58f3e36f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:40:20.194945 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-2ccc89a5-4465-499f-94a8-42da58f3e36f None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:40:20.196187 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2ccc89a5-4465-499f-94a8-42da58f3e36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:40:20.205302 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1441/2873] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:40:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:40:20.209684 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2d1284-7e86-4941-8abe-d5c5bdf65bd9 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:40:20.220310 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2d1284-7e86-4941-8abe-d5c5bdf65bd9 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:40:20.226753 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a8640d-f497-4a54-bcf2-af3dd5f1f851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:40:20.227403 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8640d-f497-4a54-bcf2-af3dd5f1f851 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:40:20.227680 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8640d-f497-4a54-bcf2-af3dd5f1f851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:40:20.228109 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8640d-f497-4a54-bcf2-af3dd5f1f851 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:40:20.228801 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a8640d-f497-4a54-bcf2-af3dd5f1f851 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 10:40:20.229100 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a8640d-f497-4a54-bcf2-af3dd5f1f851 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:40:20.230697 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a8640d-f497-4a54-bcf2-af3dd5f1f851 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:40:20.243921 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1433/2874] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 10:40:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 10:40:20.248232 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc89a5-4465-499f-94a8-42da58f3e36f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:40:20.258312 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc89a5-4465-499f-94a8-42da58f3e36f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:40:20.264756 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eec0bd-7a59-4d73-bb79-62a0dec408d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:40:20.265357 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eec0bd-7a59-4d73-bb79-62a0dec408d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:40:20.265632 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eec0bd-7a59-4d73-bb79-62a0dec408d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:40:20.265897 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eec0bd-7a59-4d73-bb79-62a0dec408d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:40:20.266547 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eec0bd-7a59-4d73-bb79-62a0dec408d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:40:20.268503 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eec0bd-7a59-4d73-bb79-62a0dec408d9 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:40:20.300445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1442/2875] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:40:20 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:40:20.304815 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a8640d-f497-4a54-bcf2-af3dd5f1f851 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:40:20.315305 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a8640d-f497-4a54-bcf2-af3dd5f1f851 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:40:20.372724 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b0d351-b216-4ba1-a2d0-def0a7932046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:40:20.373408 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0d351-b216-4ba1-a2d0-def0a7932046 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:40:20.373611 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0d351-b216-4ba1-a2d0-def0a7932046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:40:20.373884 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0d351-b216-4ba1-a2d0-def0a7932046 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:40:20.374557 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b0d351-b216-4ba1-a2d0-def0a7932046 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:40:20.376517 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b0d351-b216-4ba1-a2d0-def0a7932046 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:40:20.400901 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1434/2876] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:40:20 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:40:20.405236 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eec0bd-7a59-4d73-bb79-62a0dec408d9 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:40:20.416104 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eec0bd-7a59-4d73-bb79-62a0dec408d9 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:40:20.421575 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eb6919-cc14-4d69-921e-7d9d9d4b1014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:40:20.422194 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eb6919-cc14-4d69-921e-7d9d9d4b1014 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:40:20.422473 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eb6919-cc14-4d69-921e-7d9d9d4b1014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:40:20.422744 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eb6919-cc14-4d69-921e-7d9d9d4b1014 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:40:20.423481 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eb6919-cc14-4d69-921e-7d9d9d4b1014 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:40:20.425095 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eb6919-cc14-4d69-921e-7d9d9d4b1014 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:40:20.435695 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1443/2877] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:40:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:40:20.439738 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b0d351-b216-4ba1-a2d0-def0a7932046 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:40:20.450236 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b0d351-b216-4ba1-a2d0-def0a7932046 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:40:20.455923 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62903408-7417-4757-b9d6-7a36e436f295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:40:20.456538 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62903408-7417-4757-b9d6-7a36e436f295 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:40:20.456823 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62903408-7417-4757-b9d6-7a36e436f295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:40:20.457114 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62903408-7417-4757-b9d6-7a36e436f295 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:40:20.457796 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62903408-7417-4757-b9d6-7a36e436f295 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:40:20.459453 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62903408-7417-4757-b9d6-7a36e436f295 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:40:20.470526 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1435/2878] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:40:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:40:20.474719 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eb6919-cc14-4d69-921e-7d9d9d4b1014 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:40:20.484959 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eb6919-cc14-4d69-921e-7d9d9d4b1014 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:40:20.490445 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a0ca2-8e40-436c-8c0f-ffb0e34a1d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:40:20.491155 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a0ca2-8e40-436c-8c0f-ffb0e34a1d52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:40:20.491417 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a0ca2-8e40-436c-8c0f-ffb0e34a1d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:40:20.491690 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a0ca2-8e40-436c-8c0f-ffb0e34a1d52 admin admin] PATH_INFO: `/v3/projects/28cf7e20408a43c3bff9f70ad661d638/users/c93e81531ea54583b11f5a6c107c9166/roles/c4ba94fa4bd34a09ba87d357142d9db8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:40:20.509336 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8a0ca2-8e40-436c-8c0f-ffb0e34a1d52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28cf7e20408a43c3bff9f70ad661d638, user_id=c93e81531ea54583b11f5a6c107c9166, role_id=c4ba94fa4bd34a09ba87d357142d9db8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:40:20.512039 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8a0ca2-8e40-436c-8c0f-ffb0e34a1d52 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:40:20.528668 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1444/2879] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:40:20 2022] PUT /identity/v3/projects/28cf7e20408a43c3bff9f70ad661d638/users/c93e81531ea54583b11f5a6c107c9166/roles/c4ba94fa4bd34a09ba87d357142d9db8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:40:20.533819 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0452205-949b-47ca-9aa3-d0c4cf2e7a2d None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:40:20.534106 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0452205-949b-47ca-9aa3-d0c4cf2e7a2d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:40:20.534405 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0452205-949b-47ca-9aa3-d0c4cf2e7a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:40:20.553183 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-d0452205-949b-47ca-9aa3-d0c4cf2e7a2d None None] MFA Rules not processed for user `898d0645ae1e477983200e39184fa9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:40:20.554341 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d0452205-949b-47ca-9aa3-d0c4cf2e7a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:40:20.617410 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1436/2880] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:40:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:40:20.627474 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a0ca2-8e40-436c-8c0f-ffb0e34a1d52 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:40:20.629651 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7d8a0ca2-8e40-436c-8c0f-ffb0e34a1d52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:40:20.656284 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:40:20.694241 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a0ca2-8e40-436c-8c0f-ffb0e34a1d52 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:40:20.698483 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1ff55-f45b-4e43-9d6e-1c991242e9af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:40:20.699115 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1ff55-f45b-4e43-9d6e-1c991242e9af service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:40:20.699393 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1ff55-f45b-4e43-9d6e-1c991242e9af service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:40:20.699663 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1ff55-f45b-4e43-9d6e-1c991242e9af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:40:20.709479 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a1ff55-f45b-4e43-9d6e-1c991242e9af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:40:20.710941 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a1ff55-f45b-4e43-9d6e-1c991242e9af service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:40:20.720958 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1445/2881] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:40:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:40:22.593841 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0452205-949b-47ca-9aa3-d0c4cf2e7a2d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:40:22.595851 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d0452205-949b-47ca-9aa3-d0c4cf2e7a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:40:22.629736 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:40:22.674376 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0452205-949b-47ca-9aa3-d0c4cf2e7a2d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:40:22.679789 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3749d2a-e6db-4e58-a1bb-efff58f362ed service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8743f5bb0be54d57a957617636b0b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:40:22.680527 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3749d2a-e6db-4e58-a1bb-efff58f362ed service placement] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:40:22.680954 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3749d2a-e6db-4e58-a1bb-efff58f362ed service placement] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:40:22.681464 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3749d2a-e6db-4e58-a1bb-efff58f362ed service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:40:22.683581 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b3749d2a-e6db-4e58-a1bb-efff58f362ed service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:40:22.696394 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3749d2a-e6db-4e58-a1bb-efff58f362ed service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:40:22.697667 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3749d2a-e6db-4e58-a1bb-efff58f362ed service placement] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:40:22.709055 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1437/2882] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:40:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:40:25.026499 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812fa84-1595-4fab-9e0e-1ec9436cdd62 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:40:25.027076 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812fa84-1595-4fab-9e0e-1ec9436cdd62 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:40:25.027076 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812fa84-1595-4fab-9e0e-1ec9436cdd62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:40:25.059972 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e812fa84-1595-4fab-9e0e-1ec9436cdd62 None None] MFA Rules not processed for user `c93e81531ea54583b11f5a6c107c9166`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:40:25.061537 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e812fa84-1595-4fab-9e0e-1ec9436cdd62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:40:25.135388 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1446/2883] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:40:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:40:25.140089 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3749d2a-e6db-4e58-a1bb-efff58f362ed service placement] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:40:25.152234 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3749d2a-e6db-4e58-a1bb-efff58f362ed service placement] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:40:25.157097 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3ad0b8-885f-40a1-976b-58441c529c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:40:25.157731 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ad0b8-885f-40a1-976b-58441c529c9a admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:40:25.158009 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ad0b8-885f-40a1-976b-58441c529c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:40:25.158279 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ad0b8-885f-40a1-976b-58441c529c9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:40:25.159096 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3ad0b8-885f-40a1-976b-58441c529c9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:40:25.161034 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3ad0b8-885f-40a1-976b-58441c529c9a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:40:25.192715 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1438/2884] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 10:40:25 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 10:40:25.197176 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e812fa84-1595-4fab-9e0e-1ec9436cdd62 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:40:25.208623 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e812fa84-1595-4fab-9e0e-1ec9436cdd62 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:40:25.269690 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7af5d27-6df2-40e7-a76d-9fbaddde1ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:40:25.270212 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af5d27-6df2-40e7-a76d-9fbaddde1ffb admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:40:25.270492 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af5d27-6df2-40e7-a76d-9fbaddde1ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:40:25.270762 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af5d27-6df2-40e7-a76d-9fbaddde1ffb admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:40:25.271570 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7af5d27-6df2-40e7-a76d-9fbaddde1ffb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:40:25.273572 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7af5d27-6df2-40e7-a76d-9fbaddde1ffb admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:40:25.297000 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1447/2885] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 10:40:25 2022] POST /identity/v3/users => generated 296 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 10:40:25.301226 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3ad0b8-885f-40a1-976b-58441c529c9a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:40:25.312328 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3ad0b8-885f-40a1-976b-58441c529c9a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:40:25.317978 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736be047-5e39-4473-baa1-79e5be4be056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:40:25.318594 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736be047-5e39-4473-baa1-79e5be4be056 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:40:25.318890 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736be047-5e39-4473-baa1-79e5be4be056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:40:25.319250 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736be047-5e39-4473-baa1-79e5be4be056 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:40:25.319932 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736be047-5e39-4473-baa1-79e5be4be056 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:40:25.321621 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736be047-5e39-4473-baa1-79e5be4be056 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:40:25.331947 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1439/2886] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:40:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:40:25.336390 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7af5d27-6df2-40e7-a76d-9fbaddde1ffb admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:40:25.347255 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7af5d27-6df2-40e7-a76d-9fbaddde1ffb admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:40:25.352836 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3853a5-21ce-4dfc-97b8-11efac71394c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:40:25.353519 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3853a5-21ce-4dfc-97b8-11efac71394c admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:40:25.353798 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3853a5-21ce-4dfc-97b8-11efac71394c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:40:25.354074 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3853a5-21ce-4dfc-97b8-11efac71394c admin admin] PATH_INFO: `/v3/projects/e6d6f8295a654f288fcede13d924e548/users/c32eddf47b6f409bbcab8527002d26e2/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:40:25.369579 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3853a5-21ce-4dfc-97b8-11efac71394c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6d6f8295a654f288fcede13d924e548, user_id=c32eddf47b6f409bbcab8527002d26e2, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:40:25.372274 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3853a5-21ce-4dfc-97b8-11efac71394c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:40:25.388781 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1448/2887] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:40:25 2022] PUT /identity/v3/projects/e6d6f8295a654f288fcede13d924e548/users/c32eddf47b6f409bbcab8527002d26e2/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:40:25.392869 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736be047-5e39-4473-baa1-79e5be4be056 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:40:25.404037 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736be047-5e39-4473-baa1-79e5be4be056 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:40:25.466566 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c61b6-e14c-4ad2-8452-a570d5887c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:40:25.467252 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c61b6-e14c-4ad2-8452-a570d5887c43 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:40:25.467564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c61b6-e14c-4ad2-8452-a570d5887c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:40:25.467839 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c61b6-e14c-4ad2-8452-a570d5887c43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:40:25.468541 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1c61b6-e14c-4ad2-8452-a570d5887c43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:40:25.470144 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1c61b6-e14c-4ad2-8452-a570d5887c43 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:40:25.480498 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1440/2888] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 10:40:25 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 10:40:25.484681 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3853a5-21ce-4dfc-97b8-11efac71394c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:40:25.495635 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3853a5-21ce-4dfc-97b8-11efac71394c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:40:25.500513 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c391bd-4d60-4637-bc2c-ff8d1fa6d2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:40:25.501192 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c391bd-4d60-4637-bc2c-ff8d1fa6d2d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:40:25.501470 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c391bd-4d60-4637-bc2c-ff8d1fa6d2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:40:25.501741 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c391bd-4d60-4637-bc2c-ff8d1fa6d2d0 admin admin] PATH_INFO: `/v3/projects/e6d6f8295a654f288fcede13d924e548/users/c32eddf47b6f409bbcab8527002d26e2/roles/daef89adcaeb48a382a0bcb03340ccbb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:40:25.504105 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c391bd-4d60-4637-bc2c-ff8d1fa6d2d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6d6f8295a654f288fcede13d924e548, user_id=c32eddf47b6f409bbcab8527002d26e2, role_id=daef89adcaeb48a382a0bcb03340ccbb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:40:25.506737 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c391bd-4d60-4637-bc2c-ff8d1fa6d2d0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:40:25.517713 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1449/2889] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 10:40:25 2022] PUT /identity/v3/projects/e6d6f8295a654f288fcede13d924e548/users/c32eddf47b6f409bbcab8527002d26e2/roles/daef89adcaeb48a382a0bcb03340ccbb => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:40:29.871766 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6d7fc-9a32-4a98-93e0-06064a4d4e9c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:40:29.872262 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6d7fc-9a32-4a98-93e0-06064a4d4e9c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:40:29.872262 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6d7fc-9a32-4a98-93e0-06064a4d4e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:40:29.928734 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-6bc6d7fc-9a32-4a98-93e0-06064a4d4e9c None None] MFA Rules not processed for user `c32eddf47b6f409bbcab8527002d26e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:40:29.929943 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6bc6d7fc-9a32-4a98-93e0-06064a4d4e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:40:30.022030 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1441/2890] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:40:29 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:40:30.040306 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c391bd-4d60-4637-bc2c-ff8d1fa6d2d0 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:40:30.043113 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-49c391bd-4d60-4637-bc2c-ff8d1fa6d2d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:40:30.050742 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:40:30.122491 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c391bd-4d60-4637-bc2c-ff8d1fa6d2d0 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:40:30.128394 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375d35c2-a4c9-4305-b093-5e528947a938 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:40:30.129025 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d35c2-a4c9-4305-b093-5e528947a938 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:40:30.129299 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d35c2-a4c9-4305-b093-5e528947a938 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:40:30.129566 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d35c2-a4c9-4305-b093-5e528947a938 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:40:30.191929 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375d35c2-a4c9-4305-b093-5e528947a938 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:40:30.193539 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375d35c2-a4c9-4305-b093-5e528947a938 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:40:30.203131 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1450/2891] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:40:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:40:40.018967 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=96,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:40:40.200046 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=54 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:40:51.882827 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e3086-2688-462b-b9f7-21e101600d04 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:40:51.883621 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e3086-2688-462b-b9f7-21e101600d04 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:40:51.883621 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e3086-2688-462b-b9f7-21e101600d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:40:51.906262 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-af6e3086-2688-462b-b9f7-21e101600d04 None None] MFA Rules not processed for user `c93e81531ea54583b11f5a6c107c9166`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:40:51.907512 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-af6e3086-2688-462b-b9f7-21e101600d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:40:51.915740 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1442/2892] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:40:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:41:01.900604 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:41:08.368954 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375d35c2-a4c9-4305-b093-5e528947a938 service nova] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:41:08.370569 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-375d35c2-a4c9-4305-b093-5e528947a938 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:41:08.432862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375d35c2-a4c9-4305-b093-5e528947a938 service nova] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:41:08.437013 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f7b06e-4913-466f-912e-3f45c2aafa05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:41:08.437840 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f7b06e-4913-466f-912e-3f45c2aafa05 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:41:08.438115 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f7b06e-4913-466f-912e-3f45c2aafa05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:41:08.438384 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f7b06e-4913-466f-912e-3f45c2aafa05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:41:08.447791 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f7b06e-4913-466f-912e-3f45c2aafa05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:41:08.449222 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f7b06e-4913-466f-912e-3f45c2aafa05 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:41:08.458634 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1451/2893] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:41:08 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:41:18.454782 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=26 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:42:32.363162 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6e3086-2688-462b-b9f7-21e101600d04 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:42:32.375889 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6e3086-2688-462b-b9f7-21e101600d04 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:42:32.380506 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb12e7e2-8511-48ba-87d4-4221d96b7d1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:42:32.381129 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb12e7e2-8511-48ba-87d4-4221d96b7d1b service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:42:32.381414 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb12e7e2-8511-48ba-87d4-4221d96b7d1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:42:32.381681 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb12e7e2-8511-48ba-87d4-4221d96b7d1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:42:32.383737 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fb12e7e2-8511-48ba-87d4-4221d96b7d1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:42:32.393912 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb12e7e2-8511-48ba-87d4-4221d96b7d1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:42:32.395397 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb12e7e2-8511-48ba-87d4-4221d96b7d1b service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:42:32.403693 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1443/2894] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:42:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:42:42.400011 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=6 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:43:24.255668 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f7b06e-4913-466f-912e-3f45c2aafa05 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:43:24.268123 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f7b06e-4913-466f-912e-3f45c2aafa05 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:43:24.275521 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264408be-ce03-4262-8266-958d98de2c32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:43:24.276414 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264408be-ce03-4262-8266-958d98de2c32 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:43:24.276755 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264408be-ce03-4262-8266-958d98de2c32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:43:24.277086 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264408be-ce03-4262-8266-958d98de2c32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:43:24.279913 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-264408be-ce03-4262-8266-958d98de2c32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:43:24.297470 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264408be-ce03-4262-8266-958d98de2c32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:43:24.298960 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264408be-ce03-4262-8266-958d98de2c32 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:43:24.309893 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1452/2895] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:43:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:43:25.511354 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb12e7e2-8511-48ba-87d4-4221d96b7d1b service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:43:25.524848 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb12e7e2-8511-48ba-87d4-4221d96b7d1b service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:43:25.529323 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc5ccb-d6de-4a74-86c0-c7fd429dc96c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:43:25.529936 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc5ccb-d6de-4a74-86c0-c7fd429dc96c service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:43:25.530207 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc5ccb-d6de-4a74-86c0-c7fd429dc96c service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:43:25.530460 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc5ccb-d6de-4a74-86c0-c7fd429dc96c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:43:25.532439 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a9cc5ccb-d6de-4a74-86c0-c7fd429dc96c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:43:25.546456 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cc5ccb-d6de-4a74-86c0-c7fd429dc96c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:43:25.547917 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cc5ccb-d6de-4a74-86c0-c7fd429dc96c service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:43:25.558973 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1444/2896] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:43:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:43:27.198501 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264408be-ce03-4262-8266-958d98de2c32 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:43:27.200264 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-264408be-ce03-4262-8266-958d98de2c32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:43:27.216334 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264408be-ce03-4262-8266-958d98de2c32 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:43:27.222850 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca76d5d6-ac1f-4b12-bbe5-2f55393497fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:43:27.223623 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca76d5d6-ac1f-4b12-bbe5-2f55393497fb service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:43:27.224022 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca76d5d6-ac1f-4b12-bbe5-2f55393497fb service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:43:27.224300 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca76d5d6-ac1f-4b12-bbe5-2f55393497fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:43:27.226581 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ca76d5d6-ac1f-4b12-bbe5-2f55393497fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:43:27.242043 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca76d5d6-ac1f-4b12-bbe5-2f55393497fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:43:27.244340 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca76d5d6-ac1f-4b12-bbe5-2f55393497fb service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:43:27.262844 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1453/2897] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:43:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:43:29.053641 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0edbd2-35c7-4811-81a9-656b4b0f3bf8 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:43:29.054210 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0edbd2-35c7-4811-81a9-656b4b0f3bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:43:29.054210 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0edbd2-35c7-4811-81a9-656b4b0f3bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:43:29.073724 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-7e0edbd2-35c7-4811-81a9-656b4b0f3bf8 None None] MFA Rules not processed for user `c93e81531ea54583b11f5a6c107c9166`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:43:29.075020 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7e0edbd2-35c7-4811-81a9-656b4b0f3bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:43:29.082275 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1445/2898] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:43:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:43:29.423539 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca76d5d6-ac1f-4b12-bbe5-2f55393497fb service nova] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:43:29.438204 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca76d5d6-ac1f-4b12-bbe5-2f55393497fb service nova] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:43:29.443859 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c6e67-9ed7-449d-b87a-bde07d7fa916 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:43:29.444732 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c6e67-9ed7-449d-b87a-bde07d7fa916 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:43:29.445106 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c6e67-9ed7-449d-b87a-bde07d7fa916 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:43:29.445474 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c6e67-9ed7-449d-b87a-bde07d7fa916 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:43:29.447546 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-255c6e67-9ed7-449d-b87a-bde07d7fa916 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:43:29.457725 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255c6e67-9ed7-449d-b87a-bde07d7fa916 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:43:29.459224 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255c6e67-9ed7-449d-b87a-bde07d7fa916 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:43:29.467777 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1454/2899] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:43:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:43:29.961913 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0edbd2-35c7-4811-81a9-656b4b0f3bf8 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:43:29.963773 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7e0edbd2-35c7-4811-81a9-656b4b0f3bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:43:30.041202 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0edbd2-35c7-4811-81a9-656b4b0f3bf8 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:43:30.046460 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88fce75-250f-4844-b984-f7f51469e4b5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8743f5bb0be54d57a957617636b0b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:43:30.047119 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88fce75-250f-4844-b984-f7f51469e4b5 service placement] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:43:30.047397 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88fce75-250f-4844-b984-f7f51469e4b5 service placement] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:43:30.047667 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88fce75-250f-4844-b984-f7f51469e4b5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:43:30.049645 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c88fce75-250f-4844-b984-f7f51469e4b5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:43:30.061693 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88fce75-250f-4844-b984-f7f51469e4b5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:43:30.063299 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88fce75-250f-4844-b984-f7f51469e4b5 service placement] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:43:30.073710 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1446/2900] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:43:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:43:39.463815 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=18 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:43:40.069283 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=41 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:43:44.444515 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c6e67-9ed7-449d-b87a-bde07d7fa916 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:43:44.460174 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c6e67-9ed7-449d-b87a-bde07d7fa916 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:43:44.466832 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a8ff18-d1a2-40fc-a5b9-b0273b26a1bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:43:44.467577 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a8ff18-d1a2-40fc-a5b9-b0273b26a1bb service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:43:44.467862 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a8ff18-d1a2-40fc-a5b9-b0273b26a1bb service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:43:44.468138 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a8ff18-d1a2-40fc-a5b9-b0273b26a1bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:43:44.470003 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d8a8ff18-d1a2-40fc-a5b9-b0273b26a1bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:43:44.483757 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a8ff18-d1a2-40fc-a5b9-b0273b26a1bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:43:44.485374 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a8ff18-d1a2-40fc-a5b9-b0273b26a1bb service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:43:44.497993 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1455/2901] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:43:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:43:54.492806 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=6 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:45:29.624061 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88fce75-250f-4844-b984-f7f51469e4b5 service placement] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:45:29.684591 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88fce75-250f-4844-b984-f7f51469e4b5 service placement] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:45:29.691480 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935bb820-affc-4d3c-a20c-6f4f306646fe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8743f5bb0be54d57a957617636b0b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:45:29.692105 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935bb820-affc-4d3c-a20c-6f4f306646fe service placement] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:45:29.692406 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935bb820-affc-4d3c-a20c-6f4f306646fe service placement] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:45:29.692698 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935bb820-affc-4d3c-a20c-6f4f306646fe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:45:29.701986 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935bb820-affc-4d3c-a20c-6f4f306646fe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:45:29.704150 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935bb820-affc-4d3c-a20c-6f4f306646fe service placement] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:45:29.714250 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1447/2902] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:45:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:45:39.710117 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=27 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:45:39.731190 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a8ff18-d1a2-40fc-a5b9-b0273b26a1bb service nova] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:45:39.732869 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d8a8ff18-d1a2-40fc-a5b9-b0273b26a1bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:45:39.849656 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a8ff18-d1a2-40fc-a5b9-b0273b26a1bb service nova] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:45:39.854212 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982163b-51e3-4792-8209-224cd039d823 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f70369fd5b334d888de0c091d8885eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:45:39.854850 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982163b-51e3-4792-8209-224cd039d823 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:45:39.855162 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982163b-51e3-4792-8209-224cd039d823 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:45:39.855419 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982163b-51e3-4792-8209-224cd039d823 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:45:39.865140 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a982163b-51e3-4792-8209-224cd039d823 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:45:39.866530 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a982163b-51e3-4792-8209-224cd039d823 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:45:39.875786 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1456/2903] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:45:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:45:49.871640 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=48 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:45:51.002456 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aacddb-633b-4d5f-ab80-fb981382d275 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:45:51.003109 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aacddb-633b-4d5f-ab80-fb981382d275 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:45:51.003109 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aacddb-633b-4d5f-ab80-fb981382d275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:45:51.023967 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-12aacddb-633b-4d5f-ab80-fb981382d275 None None] MFA Rules not processed for user `c93e81531ea54583b11f5a6c107c9166`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 10:45:51.025218 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-12aacddb-633b-4d5f-ab80-fb981382d275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:45:51.033586 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1448/2904] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 10:45:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 10:46:01.018139 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:47:40.265446 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982163b-51e3-4792-8209-224cd039d823 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:47:40.288049 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982163b-51e3-4792-8209-224cd039d823 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:47:40.293592 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369eaffa-e395-43b7-905d-8a9c83d4449d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:47:40.294211 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369eaffa-e395-43b7-905d-8a9c83d4449d service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:47:40.294492 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369eaffa-e395-43b7-905d-8a9c83d4449d service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:47:40.294764 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369eaffa-e395-43b7-905d-8a9c83d4449d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:47:40.303676 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369eaffa-e395-43b7-905d-8a9c83d4449d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:47:40.305121 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369eaffa-e395-43b7-905d-8a9c83d4449d service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:47:40.313367 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1457/2905] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:47:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:47:50.310058 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=10 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:47:59.070490 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aacddb-633b-4d5f-ab80-fb981382d275 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:47:59.103404 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aacddb-633b-4d5f-ab80-fb981382d275 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:47:59.109512 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb5b3c8-1e1b-4ca3-90db-cdfa97c40492 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b938f1f41db94b0eafe95bc8eeaae175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:47:59.110165 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5b3c8-1e1b-4ca3-90db-cdfa97c40492 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:47:59.110506 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5b3c8-1e1b-4ca3-90db-cdfa97c40492 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:47:59.110782 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5b3c8-1e1b-4ca3-90db-cdfa97c40492 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:47:59.112743 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-bbb5b3c8-1e1b-4ca3-90db-cdfa97c40492 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:47:59.145569 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb5b3c8-1e1b-4ca3-90db-cdfa97c40492 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:47:59.147501 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb5b3c8-1e1b-4ca3-90db-cdfa97c40492 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:47:59.169787 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1449/2906] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:47:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:48:09.164097 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=18 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:48:12.684893 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369eaffa-e395-43b7-905d-8a9c83d4449d service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:48:12.697630 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369eaffa-e395-43b7-905d-8a9c83d4449d service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:48:12.702040 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f69c94-5626-43c5-8a0b-99dc67c1a474 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7deb94069a449269cb03406232174d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:48:12.702655 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f69c94-5626-43c5-8a0b-99dc67c1a474 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:48:12.702928 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f69c94-5626-43c5-8a0b-99dc67c1a474 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:48:12.703251 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f69c94-5626-43c5-8a0b-99dc67c1a474 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:48:12.785732 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f69c94-5626-43c5-8a0b-99dc67c1a474 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:48:12.787818 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f69c94-5626-43c5-8a0b-99dc67c1a474 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:48:12.808222 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1458/2907] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:48:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:48:15.971937 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb5b3c8-1e1b-4ca3-90db-cdfa97c40492 service nova] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:48:15.983929 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb5b3c8-1e1b-4ca3-90db-cdfa97c40492 service nova] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:48:15.989659 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279b6b3a-80d7-49ff-82cc-99bcb148fa10 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8743f5bb0be54d57a957617636b0b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:48:15.990288 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b6b3a-80d7-49ff-82cc-99bcb148fa10 service placement] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:48:15.990548 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b6b3a-80d7-49ff-82cc-99bcb148fa10 service placement] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:48:15.990829 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b6b3a-80d7-49ff-82cc-99bcb148fa10 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:48:15.992723 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-279b6b3a-80d7-49ff-82cc-99bcb148fa10 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:48:16.019836 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279b6b3a-80d7-49ff-82cc-99bcb148fa10 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:48:16.021657 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279b6b3a-80d7-49ff-82cc-99bcb148fa10 service placement] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:48:16.043254 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1450/2908] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:48:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:48:18.900654 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f69c94-5626-43c5-8a0b-99dc67c1a474 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:48:18.913807 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f69c94-5626-43c5-8a0b-99dc67c1a474 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:48:18.919359 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c5cb4f-095c-490a-8d4e-e8ecb2d517bf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8743f5bb0be54d57a957617636b0b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:48:18.919998 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5cb4f-095c-490a-8d4e-e8ecb2d517bf service placement] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:48:18.920279 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5cb4f-095c-490a-8d4e-e8ecb2d517bf service placement] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:48:18.920545 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5cb4f-095c-490a-8d4e-e8ecb2d517bf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:48:18.922436 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-05c5cb4f-095c-490a-8d4e-e8ecb2d517bf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:48:18.935339 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c5cb4f-095c-490a-8d4e-e8ecb2d517bf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:48:18.936986 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c5cb4f-095c-490a-8d4e-e8ecb2d517bf service placement] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:48:18.948108 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1459/2909] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:48:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:48:18.976872 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279b6b3a-80d7-49ff-82cc-99bcb148fa10 service placement] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:48:18.988878 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279b6b3a-80d7-49ff-82cc-99bcb148fa10 service placement] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:48:18.995157 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcbe73d-8cc3-4d09-837a-da500d7c88e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:48:18.995897 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcbe73d-8cc3-4d09-837a-da500d7c88e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:48:18.996017 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcbe73d-8cc3-4d09-837a-da500d7c88e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:48:18.996407 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcbe73d-8cc3-4d09-837a-da500d7c88e7 admin admin] PATH_INFO: `/v3/users/c93e81531ea54583b11f5a6c107c9166` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:48:18.997570 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcbe73d-8cc3-4d09-837a-da500d7c88e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c93e81531ea54583b11f5a6c107c9166)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:48:18.999564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcbe73d-8cc3-4d09-837a-da500d7c88e7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:48:19.049519 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1dcbe73d-8cc3-4d09-837a-da500d7c88e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c93e81531ea54583b11f5a6c107c9166'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:48:19.058413 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1dcbe73d-8cc3-4d09-837a-da500d7c88e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c93e81531ea54583b11f5a6c107c9166'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:48:19.059061 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1dcbe73d-8cc3-4d09-837a-da500d7c88e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c93e81531ea54583b11f5a6c107c9166'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:48:19.072403 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1dcbe73d-8cc3-4d09-837a-da500d7c88e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c93e81531ea54583b11f5a6c107c9166'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:48:19.073081 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1dcbe73d-8cc3-4d09-837a-da500d7c88e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c93e81531ea54583b11f5a6c107c9166'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:48:19.085564 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1451/2910] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:48:18 2022] DELETE /identity/v3/users/c93e81531ea54583b11f5a6c107c9166 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 10:48:19.272313 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c5cb4f-095c-490a-8d4e-e8ecb2d517bf service placement] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:48:19.285907 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c5cb4f-095c-490a-8d4e-e8ecb2d517bf service placement] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:48:19.349173 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cfa540-2f67-449c-8502-f47e776ef685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:48:19.349810 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cfa540-2f67-449c-8502-f47e776ef685 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:48:19.350119 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cfa540-2f67-449c-8502-f47e776ef685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:48:19.350394 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cfa540-2f67-449c-8502-f47e776ef685 admin admin] PATH_INFO: `/v3/projects/28cf7e20408a43c3bff9f70ad661d638` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:48:19.351649 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cfa540-2f67-449c-8502-f47e776ef685 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28cf7e20408a43c3bff9f70ad661d638)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:48:19.354496 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cfa540-2f67-449c-8502-f47e776ef685 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:48:19.414275 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-48cfa540-2f67-449c-8502-f47e776ef685 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28cf7e20408a43c3bff9f70ad661d638', 'request_id': 'req-48cfa540-2f67-449c-8502-f47e776ef685'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:48:19.424473 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-48cfa540-2f67-449c-8502-f47e776ef685 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28cf7e20408a43c3bff9f70ad661d638', 'request_id': 'req-48cfa540-2f67-449c-8502-f47e776ef685'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:48:19.430195 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-48cfa540-2f67-449c-8502-f47e776ef685 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28cf7e20408a43c3bff9f70ad661d638', 'request_id': 'req-48cfa540-2f67-449c-8502-f47e776ef685'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:48:19.431321 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:48:19.456029 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-48cfa540-2f67-449c-8502-f47e776ef685 admin admin] The token cache is being invalidate because project 28cf7e20408a43c3bff9f70ad661d638 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:48:19.456321 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-48cfa540-2f67-449c-8502-f47e776ef685 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:48:19.456940 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-48cfa540-2f67-449c-8502-f47e776ef685 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:48:19.458465 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1460/2911] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:48:19 2022] DELETE /identity/v3/projects/28cf7e20408a43c3bff9f70ad661d638 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:48:19.462605 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcbe73d-8cc3-4d09-837a-da500d7c88e7 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:48:19.464136 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1dcbe73d-8cc3-4d09-837a-da500d7c88e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:48:19.477440 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcbe73d-8cc3-4d09-837a-da500d7c88e7 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:48:19.534881 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9a2da0-d0bb-49ae-94f2-beff40f016bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:48:19.535590 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a2da0-d0bb-49ae-94f2-beff40f016bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:48:19.535871 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a2da0-d0bb-49ae-94f2-beff40f016bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:48:19.536128 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a2da0-d0bb-49ae-94f2-beff40f016bb admin admin] PATH_INFO: `/v3/users/c32eddf47b6f409bbcab8527002d26e2` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:48:19.537320 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9a2da0-d0bb-49ae-94f2-beff40f016bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c32eddf47b6f409bbcab8527002d26e2)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:48:19.539272 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9a2da0-d0bb-49ae-94f2-beff40f016bb admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:48:19.586161 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1a9a2da0-d0bb-49ae-94f2-beff40f016bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c32eddf47b6f409bbcab8527002d26e2'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:48:19.588532 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:48:19.594585 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1a9a2da0-d0bb-49ae-94f2-beff40f016bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c32eddf47b6f409bbcab8527002d26e2'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:48:19.595363 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1a9a2da0-d0bb-49ae-94f2-beff40f016bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c32eddf47b6f409bbcab8527002d26e2'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:48:19.608559 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1a9a2da0-d0bb-49ae-94f2-beff40f016bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c32eddf47b6f409bbcab8527002d26e2'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:48:19.609190 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1a9a2da0-d0bb-49ae-94f2-beff40f016bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c32eddf47b6f409bbcab8527002d26e2'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:48:19.621621 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1452/2912] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 10:48:19 2022] DELETE /identity/v3/users/c32eddf47b6f409bbcab8527002d26e2 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:48:19.798158 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cfa540-2f67-449c-8502-f47e776ef685 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:48:19.799642 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-48cfa540-2f67-449c-8502-f47e776ef685 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:48:19.813844 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cfa540-2f67-449c-8502-f47e776ef685 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:48:19.873393 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a57a46-8138-40b1-bc5e-93130f5998e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '898d0645ae1e477983200e39184fa9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be89c43472bc49a2bd0e46cddf175958', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:48:19.874041 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a57a46-8138-40b1-bc5e-93130f5998e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:48:19.874319 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a57a46-8138-40b1-bc5e-93130f5998e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:48:19.874589 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a57a46-8138-40b1-bc5e-93130f5998e1 admin admin] PATH_INFO: `/v3/projects/e6d6f8295a654f288fcede13d924e548` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:48:19.875777 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a57a46-8138-40b1-bc5e-93130f5998e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6d6f8295a654f288fcede13d924e548)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:48:19.877686 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a57a46-8138-40b1-bc5e-93130f5998e1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:48:19.935720 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-79a57a46-8138-40b1-bc5e-93130f5998e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6d6f8295a654f288fcede13d924e548', 'request_id': 'req-79a57a46-8138-40b1-bc5e-93130f5998e1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:48:19.944212 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-79a57a46-8138-40b1-bc5e-93130f5998e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6d6f8295a654f288fcede13d924e548', 'request_id': 'req-79a57a46-8138-40b1-bc5e-93130f5998e1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:48:19.949936 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-79a57a46-8138-40b1-bc5e-93130f5998e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6d6f8295a654f288fcede13d924e548', 'request_id': 'req-79a57a46-8138-40b1-bc5e-93130f5998e1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:48:19.960860 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-79a57a46-8138-40b1-bc5e-93130f5998e1 admin admin] The token cache is being invalidate because project e6d6f8295a654f288fcede13d924e548 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 10:48:19.961149 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-79a57a46-8138-40b1-bc5e-93130f5998e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:48:19.961766 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-79a57a46-8138-40b1-bc5e-93130f5998e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 10:48:19.963440 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1461/2913] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 10:48:19 2022] DELETE /identity/v3/projects/e6d6f8295a654f288fcede13d924e548 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 10:48:29.618145 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:DELETE=3,keystone:SELECT=12 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:48:29.958168 dv-cinder-815582-15-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=59,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 10:50:35.565664 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9a2da0-d0bb-49ae-94f2-beff40f016bb admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 10:50:35.567282 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1a9a2da0-d0bb-49ae-94f2-beff40f016bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:50:35.643293 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9a2da0-d0bb-49ae-94f2-beff40f016bb admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 10:50:35.709080 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94108cc7-3b24-44e8-896e-92f2f7d5ee55 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8743f5bb0be54d57a957617636b0b8fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36b8b505012d4d0bac54845b6799e743', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 10:50:35.709702 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94108cc7-3b24-44e8-896e-92f2f7d5ee55 service placement] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 10:50:35.709978 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94108cc7-3b24-44e8-896e-92f2f7d5ee55 service placement] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 10:50:35.710250 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94108cc7-3b24-44e8-896e-92f2f7d5ee55 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 10:50:35.712246 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-94108cc7-3b24-44e8-896e-92f2f7d5ee55 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 10:50:35.725699 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94108cc7-3b24-44e8-896e-92f2f7d5ee55 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 10:50:35.727336 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94108cc7-3b24-44e8-896e-92f2f7d5ee55 service placement] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 10:50:35.738215 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1453/2914] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 10:50:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 10:50:45.733299 dv-cinder-815582-15-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=58 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}